pantsbuild.pants 0.0.43

Release Notes

This release makes the isolated jvm compile strategy viable out-of-the-box for use with large dependency graphs. Without it, test.junit and run.jvm performance slows down significantly due to the large number of loose classfile directories.

Please try it out in your repo by grabbing a copy of pants.ini.isolated and using a command like:

./pants --config-override=pants.ini.isolated test examples/{src,tests}/{scala,java}/::

You’ll like the results. Just update your own pants.ini with the pants.ini.isolated settings to use it by default!

In the medium term, we’re interested in making the isolated strategy the default jvm compilation strategy, so your assistance and feedback is appreciated!

Special thanks to Stu Hood and Nick Howard for lots of work over the past months to get this point.

API Changes

• A uniform way of expressing Task and Subsystem dependencies. Issue #1957 RB #2653

• Remove some coverage-related options from test.junit. RB #2639

• Bump mock and six 3rdparty versions to latest RB #2633

• Re-implement suppression of output from compiler workunits RB #2590

Bugfixes

• Improved go remote library support. RB #2655

• Shorten isolation generated jar paths RB #2647

• Fix duplicate login options when publishing. RB #2560

• Fixed no attribute exception in changed goal. RB #2645

• Fix goal idea issues with mistakenly identifying a test folder as regular code, missing resources folders, and resources folders overriding code folders. RB #2046 RB #2642

New Features

• Support for running junit tests with different jvm versions. RB #2651

• Add support for jar’ing compile outputs in the isolated strategy. RB #2643

• Tests for ‘java-resoures’ and ‘java-test-resources’ in idea RB #2046 RB #2634

Small improvements, Refactoring and Tooling

• Filter zinc compilation warnings at the Reporter level RB #2656

• Update to sbt 0.13.9. RB #2629

• Speeding up jvm-platform-validate step. Issue #1972 RB #2626

• Added test that failed HTTP responses do not raise exceptions in artifact cache RB #2624 RB #2644

• Tweak to option default extraction for help display. RB #2640

• A few small install doc fixes. RB #2638

• Detect new package when doing ownership checks. RB #2637

• Use os.path.realpath on test tmp dirs to appease OSX. RB #2635

• Update the pants install documentation. #docfixit RB #2631

Release Notes

This was #docfixit week, so the release contains more doc and help improvements than usual. Thanks in particular to Benjy for continued ./pants help polish!

This release also add support for golang in the contrib/go package. Thanks to Cody Gibb and John Sirois for that work.

API Changes

• Elevate the pants version to a first class option RB #2627

• Support pants plugin resolution for easier inclusion of published plugins RB #2615 RB #2622

• Pin pex==1.0.3, alpha-sort & remove line breaks RB #2598 RB #2596

• Moved classifier from IvyArtifact to IvyModuleRef RB #2579

Bugfixes

• Ignore ‘NonfatalArtifactCacheError’ when calling the artifact cache in the background RB #2624

• Re-Add debug option to benchmark run task, complain on no jvm targets, add test RB #2619

• Fixed what_changed for removed files RB #2589

• Disable jvm-platform-analysis by default Issue #1972 RB #2618

• Fix ./pants help_advanced RB #2616

• Fix some more missing globs in build-file-rev mode. RB #2591

• Make jvm bundles output globs in filedeps with –globs. RB #2583

• Fix more realpath issues Issue #1933 RB #2582

New Features

• Allow plaintext-reporter to be able to respect a task’s –level and –colors options. RB #2580 RB #2614

• contrib/go: Support for Go RB #2544

• contrib/go: Setup a release sdist RB #2609

• contrib/go: Remote library support RB #2611 RB #2623

• contrib/go: Introduce GoDistribution RB #2595

• contrib/go: Integrate GoDistribution with GoTask RB #2600

• Add support for android compilation with contrib/scrooge RB #2553

Small improvements, Refactoring and Tooling

• Added more testimonials to the Powered By page. #docfixit RB #2625

• Fingerprint more task options; particularly scalastyle configs RB #2628

• Fingerprint jvm tools task options by default RB #2620

• Make most compile-related options advanced. #docfixit RB #2617

• Make almost all global options advanced. #docfixit RB #2602

• Improve cmd-line help output. #docfixit RB #2599

• Default -Dscala.usejavacp=true for ScalaRepl. RB #2613

• Additional Option details for the Task developers guide. #docfixit RB #2594 RB #2612

• Improve subsystem testing support in subsystem_util. RB #2603

• Cleanups to the tasks developer’s guide #docfixit RB #2594

• Add the optionable class to ScopeInfo. #docfixit RB #2588

• Add pants_plugin and contrib_plugin targets. RB #2615

Release Notes

Configuration for specifying scala/java compilation using zinc has changed in this release.

You may need to combine [compile.zinc-java] and [compile.scala] into the new section [compile.zinc]

The migrate_config tool will help you migrate your pants.ini settings for this new release. Download the pants source code and run:

./pants run migrations/options/src/python:migrate_config --  <path to your pants.ini>

API Changes

• Upgrade pex to 1.0.2. RB #2571

Bugfixes

• Fix ApacheThriftGen chroot normalization scope. RB #2568

• Fix crasher when no jvm_options are set RB #2578

• Handle recursive globs with build-file-rev RB #2572

• Fixup PythonTask chroot caching. RB #2567

New Features

• Add “omnivorous” ZincCompile to consume both java and scala sources RB #2561

Small improvements, Refactoring and Tooling

• Do fewer classpath calculations in junit_run. RB #2576

• fix misc ws issues RB #2564 RB #2557

• Resurrect the –[no-]lock global flag RB #2563

• Avoid caching volatile ~/.cache/pants/stats dir. RB #2574

• remove unused imports RB #2556

• Moved logic which validates jvm platform dependencies. RB #2565

• Bypass the pip cache when testing released sdists. RB #2555

• Add an affordance for 1 flag implying another. RB #2562

• Make artifact cache max-entries-per-target option name match its behaviour RB #2550

• Improve stats upload. RB #2554

Release Notes

The apache thrift gen for java code now runs in -strict mode by default, requiring all struct fields declare a field id. You can use the following configuration in pants.ini to retain the old default behavior and turn strict checking off:

[gen.thrift]
strict: False

The psutil dependency used by pants has been upgraded to 3.1.1. Supporting eggs have been uploaded to https://github.com/pantsbuild/cheeseshop/tree/gh-pages/third_party/python/dist. Please note that beyond this update, no further binary dependency updates will be provided at this location.

API Changes

• Integrate the Android SDK, android-library RB #2528

Bugfixes

• Guard against NoSuchProcess in the public API. RB #2551

• Fixup psutil.Process attribute accesses. RB #2549

• Removes type=Option.list from –compile-jvm-args option and –compile-scala-plugins RB #2536 RB #2547

• Prevent nailgun on nailgun violence when using symlinked java paths RB #2538

• Declaring product_types for simple_codegen_task. RB #2540

• Fix straggler usage of legacy psutil form RB #2546

New Features

• Added JvmPlatform subsystem and added platform arg to JvmTarget. RB #2494

Small improvements, Refactoring and Tooling

• Resolve targets before creating PayloadField RB #2496 RB #2536

• Upgrade psutil to 3.1.1 RB #2543

• Move thrift utils only used by scrooge to contrib/scrooge. RB #2535

• docs: add link to slackin self-invite RB #2537

• Add Clover Health to the Powered By page RB #2539

• Add Powered By page RB #2532

• Create test for java_antlr_library RB #2504

• Migrate ApacheThriftGen to SimpleCodegenTask. RB #2534

• Covert RagelGen to SimpleCodeGen. RB #2531

• Shade the Checkstyle task tool jar. RB #2533

• Support eggs for setuptools and wheel. RB #2529

API Changes

• Disallow jar_library targets without jars RB #2519

Bugfixes

• Fixup PythonChroot to ignore synthetic targets. RB #2523

• Exclude provides clauses regardless of soft_excludes RB #2524

• Fixed exclude id when name is None + added a test for excludes by just an org #1857 RB #2518

• Fixup SourceRoot to handle the buildroot. RB #2514

• Fixup SetupPy handling of exported thrift. RB #2511

New Features

• Invalidate tasks based on BinaryUtil.version. RB #2516

• Remove local cache files Issue #1762 RB #2506

• Option to expose intransitive target dependencies for the dependencies goal RB #2503

• Introduce Subsystem dependencies. RB #2509 RB #2515

Small improvements, Refactoring and Tooling

• Increase robustness of ProcessManager.terminate() in the face of zombies. RB #2513

• A global isort fix. RB #2510

Release Notes

A quick hotfix release to pick up a fix related to incorrectly specified scala targets.

API Changes

• Remove the with_description method from target. RB #2507

Bugfixes

• Handle the case where there are no classes for a target. RB #2489

New Features

None.

Small improvements, Refactoring and Tooling

• Refactor AntlrGen to use SimpleCodeGen. RB #2487

Release Notes

This is the regularly scheduled release for 7/17/2015 (slightly behind schedule!)

API Changes

• Unified support for process management, to prepare for a new daemon. RB #2490

• An iterator over Option registration args. RB #2478

• An iterator over OptionValueContainer keys. RB #2472

Bugfixes

• Correctly classify files as resources or classes RB #2488

• Fix test bugs introduced during the target cache refactor. RB #2483

• Don’t explicitly enumerate goal scopes: makes life easier for the IntelliJ pants plugin. RB #2500

New Features

• Switch almost all python tasks over to use cached chroots. RB #2486

• Add invalidation report flag to reporting subsystem. RB #2448

Small improvements, Refactoring and Tooling

• Add a note about the pantsbuild slack team. RB #2491

• Upgrade pantsbuild/pants to apache thrift 0.9.2. RB #2484

• Remove unused –lang option from protobuf_gen.py RB #2485

• Update release docs to recommend both server-login and pypi sections. RB #2481

Release Notes

This is a quick release following up on 0.0.35 to make available internal API changes made during options refactoring.

API Changes

• Improved artifact cache usability by allowing tasks to opt-in to a mode that generates and then caches a directory for each target. RB #2449 RB #2471

• Re-compute the classpath for each batch of junit tests. RB #2454

Bugfixes

• Stops unit tests in test_simple_codegen_task.py in master from failing. RB #2469

• Helpful error message when ‘sources’ is specified for jvm_binary. Issue #871 RB #2455

• Fix failure in test_execute_fail under python>=2.7.10 for test_simple_codegen_task.py. RB #2461

New Features

• Support short-form task subsystem flags. RB #2466

• Reimplement help formatting to improve clarity of both the code and output. RB #2458 RB #2464

Small improvements, Refactoring and Tooling

• Visual docsite changes RB #2463

• Fix migrate_config to detect explicit [DEFAULT]s. RB #2465

Release Notes

With this release, if you use the isolated jvm compile strategy, java annotation processers that emit java sourcefiles or classfiles will be handled correctly and the generated code will be bundled appropriately in jars. In particular, this makes libraries like Google’s AutoValue useable in a pants build. See: RB #2451.

API Changes

• Deprecate with_description. RB #2444

Bugfixes

• Fixup BuildFile must_exist logic. RB #2441

• Upgrade to pex 1.0.1. Issue #1658 RB #2438

New Features

• Add an option –main to the run.jvm task to override the specification of ‘main’ on a jvm_binary() target. RB #2442

• Add jvm_options for thrift-linter. RB #2445

• Added cwd argument to allow JavaTest targets to require particular working directories. RB #2440

Small improvements, Refactoring and Tooling

• Record all output classes for the jvm isolated compile strategy. RB #2451

• Robustify the pants ivy configuration. Issue #1779 RB #2450

• Some refactoring of global options. RB #2446

• Improved error messaging for unknown Target kwargs. RB #2443

• Remove Nailgun specific classes from zinc, since pants invokes Main directly. RB #2439

Release Notes

Configuration for specifying cache settings and jvm options for some tools have changed in this release.

The migrate_config tool will help you migrate your pants.ini settings for this new release. Download the pants source code and run:

./pants run migrations/options/src/python:migrate_config --  <path
to your pants.ini>

API Changes

• Added flags for jar sources and javadocs to export goal because Foursquare got rid of ivy goal. RB #2432

• A JVM subsystem. RB #2423

• An artifact cache subsystem. RB #2405

Bugfixes

• Change the xml report to use the fingerprint of the targets, not just their names. RB #2435

• Using linear-time BFS to sort targets topologically and group them by the type. RB #2413

• Fix isort in git hook context. RB #2430

• When using soft-excludes, ignore all target defined excludes RB #2340

• Fix bash-completion goal when run from sdist/pex. Also add tests, and beef up ci.sh & release.sh. RB #2403

• [junit tool] fix suppress output emits jibberish on console. Issue #1657 RB #2183

• In junit-runner, fix an NPE in testFailure() for different scenarios RB #2385 RB #2398 RB #2396

• Scrub timestamp from antlr generated files to have stable fp for cache RB #2382

• JVM checkstyle should obey jvm_options RB #2391

• Fix bad logger.debug call in artifact_cache.py RB #2386

• Fixed a bug where codegen would crash due to a missing flag. RB #2368

• Fixup the Git Scm detection of server_url. RB #2379

• Repair depmap –graph RB #2345

Documentation

• Documented how to enable caching for tasks. RB #2420

• Remove comments that said these classes returned something. RB #2419

• Publishing doc fixes RB #2407

• Bad rst now fails the MarkdownToHtml task. RB #2394

• Add a CONTRIBUTORS maintenance script. RB #2377 RB #2378

• typo in the changelog for 0.0.33 release, fixed formatting of globs and rglobs RB #2376

• Documentation update for debugging a JVM tool RB #2365

New Features

• Add log capture to isolated zinc compiles RB #2404 RB #2415

• Add support for restricting push remotes. RB #2383

• Ensure caliper is shaded in bench, add bench desc, use RUN so that output is printed RB #2353

Small improvements, Refactoring and Tooling

• Enhance the error output in simple_codegen_task.py when unable to generate target(s) RB #2427

• Add a get_rank() method to OptionValueContainer. RB #2431

• Pass jvm_options to scalastyle RB #2428

• Kill custom repos and cross-platform pex setup. RB #2402

• Add debugging for problem with invalidation and using stale report file in ivy resolve. Issue #1747 RB #2424

• Enabled caching for scalastyle and checkstyle RB #2416 RB #2414

• Make sure all Task mixins are on the left. RB #2421

• Adds a more verbose description of tests when running the -per-test-timer command. (Junit) RB #2418 RB #2408

• Re-add support for reading from a local .m2 directory RB #2409

• Replace a few references to basestring with six. RB #2410

• Promote PANTS_DEV=1 to the only ./pants mode. RB #2401

• Add task meter to protoc step in codegen RB #2392

• Simplify known scopes computation. RB #2389

• Robustify the release process. RB #2388

• A common base class for things that can register options. RB #2387

• Fixed the error messages in assert_list(). RB #2370

• Simplify subsystem option scoping. RB #2380

Release Notes

The migrate config tool will help you migrate your pants.ini settings for this new release. Download the pants source code and run:

./pants run migrations/options/src/python:migrate_config --  <path
to your pants.ini>

Folks who use a custom ivysettings.xml but have no ivy.ivy_settings option defined in pants.ini pointing to it must now add one like so:

[ivy]
ivy_settings: %(pants_supportdir)s/ivy/ivysettings.xml

API Changes

• Removed –project-info flag from depmap goal RB #2363

• Deprecate PytestRun env vars. RB #2299

• Add Subsystems for options that live outside a single task, use them to replace config settings in pants.ini RB #2288 RB #2276 RB #2226 RB #2176 RB #2174 RB #2139 RB #2122 RB #2100 RB #2081 RB #2063

• Read backend and bootstrap BUILD file settings from options instead of config. RB #2229

• Migrating internal tools into the pants repo and renaming to org.pantsbuild RB #2278 RB #2211 RB #2207 RB #2205 RB #2186 RB #2195 RB #2193 RB #2192 RB #2191 RB #2191 RB #2137 RB #2071 RB #2043

• Kill scala specs support. RB #2208

• Use the default ivysettings.xml provided by ivy. RB #2204

• Eliminate the globs.__sub__ use in option package. RB #2082 RB #2197

• Kill obsolete global publish.properties file. RB #994 RB #2069

• Upgrade zinc to latest for perf wins. RB #2355 RB #2194 RB #2168 RB #2154 RB #2154 RB #2149 RB #2125

• Migrate jar_publish config scope. RB #2175

• Add a version number to the export format and a page with some documentation. RB #2162

• Make exclude_target_regexp option recursive RB #2136

• Kill pantsbuild dependence on maven.twttr.com. RB #2019

• Fold PythonTestBuilder into the PytestRun task. RB #1993

Bugfixes

• Fixed errors in how arguments are passed to wire_gen. RB #2354

• Compute exclude_patterns first when unpacking jars RB #2352

• Add INDEX.LIST to as a Skip JarRule when creating a fat jar RB #2342

• wrapped-globs: make rglobs output git-compatible RB #2332

• Add a coherent error message when scrooge has no sources. RB #2329

• Only run junit when there are junit_test targets in the graph. RB #2291

• Fix bootstrap local cache. RB #2336

• Added a hash to a jar name for a bootstrapped jvm tool RB #2334

• Raise TaskError to exit non-zero if jar-tool fails RB #2150

• Fix java zinc isolated compile analysis corruption described github issue #1626 RB #2325

• Upstream analysis fix RB #2312

• Two changes that affect invalidation and artifact caching. RB #2269

• Add java_thrift_library fingerprint strategy RB #2265

• Moved creation of per test data to testStarted method. RB #2257

• Updated zinc to use sbt 0.13.8 and new java compilers that provide a proper log level with their output. RB #2248

• Apply excludes consistently across classpaths RB #2247

• Put all extra classpath elements (e.g., plugins) at the end (scala compile) RB #2210

• Fix missing import in git.py RB #2202

• Move a comment to work around a pytest bug. RB #2201

• More fixes for working with classifiers on jars. Issue #1489 RB #2163

• Have ConsoleRunner halt(1) on exit(x) RB #2180

• Fix scm_build_file in symlinked directories RB #2152 RB #2157

• Added support for the ivy cache being under a symlink’ed dir RB #2085 RB #2129 RB #2148

• Make subclasses of ChangedTargetTask respect spec_excludes RB #2146

• propagate keyboard interrupts from worker threads RB #2143

• Only add resources to the relevant target RB #2103 RB #2130

• Cleanup analysis left behind from failed isolation compiles RB #2127

• test glob operators, fix glob + error RB #2104

• Wrap lock around nailgun spawning to protect against worker threads racing to spawn servers RB #2102

• Force some files to be treated as binary. RB #2099

• Convert JarRule and JarRules to use Payload to help fingerprint its configuration RB #2096

• Fix ./pants server output RB #2067

• Fix issue with isolated strategy and sources owned by multiple targets RB #2061

• Handle broken resource mapping files (by throwing exceptions). RB #2038

• Change subproc sigint handler to exit more cleanly RB #2024

• Include classifier in JarDependency equality / hashing RB #2029

• Migrating more data to payload fields in jvm_app and jvm_binary targets RB #2011

• Fix ivy_resolve message: Missing expected ivy output file …/.ivy2/pants/internal-…-default.xml RB #2015

• Fix ignored invalidation data in ScalaCompile RB #2018

• Don’t specify the jmake depfile if it doesn’t exist RB #2009 RB #2012

• Force java generation on for protobuf_gen, get rid of spurious warning RB #1994

• Fix typo in ragel-gen entries (migrate-config) RB #1995

• Fix include dependees options. RB #1760

Documentation

• Be explicit that pants requires python 2.7.x to run. RB #2343

• Update documentation on how to develop and document a JVM tool used by Pants RB #2318

• Updates to changelog since 0.0.32 in preparation for next release. RB #2294

• Document the pantsbuild jvm tool release process. RB #2289

• Fix publishing docs for new ‘publish.jar’ syntax RB #2255

• Example configuration for the isolated strategy. RB #2185

• doc: uploading timing stats RB #1700

• Add robots.txt to exclude crawlers from walking a ‘staging’ test publishing dir RB #2072

• Add a note indicating that pants bootstrap requires a compiler RB #2057

• Fix docs to mention automatic excludes. RB #2014

New Features

• Add a global –tag option to filter targets based on their tags. RB #2362

• Add support for ServiceLoader service providers. RB #2331

• Implemented isolated code-generation strategy for simple_codegen_task. RB #2322

• Add options for specifying python cache dirs. RB #2320

• bash autocompletion support RB #2307 RB #2326

• Invoke jvm doc tools via java. RB #2313

• Add -log-filter option to the zinc task RB #2315

• Adds a product to bundle_create RB #2254

• Add flag to disable automatic excludes RB #2252

• Find java distributions in well known locations. RB #2242

• Added information about excludes to export goal RB #2238

• In process java compilation in Zinc #1555 RB #2206

• Add support for extra publication metadata. RB #2184 RB #2240

• Extract the android plugin as an sdist. RB #2249

• Adds optional output during zinc compilation. RB #2233

• Jvm Tools release process RB #2292

• Make it possible to create xml reports and output to console at the same time from ConsoleRunner. RB #2183

• Adding a product to binary_create so that we can depend on it in an external plugin. RB #2172

• Publishing to Maven Central RB #2068 RB #2188

• Provide global option to look up BUILD files in git history RB #2121 RB #2164

• Compile Java with Zinc RB #2156

• Add BuildFileManipulator implementation and tests to contrib RB #977

• Add option to suppress printing the changelog during publishing RB #2140

• Filtering by targets’ tags RB #2106

• Adds the ability to specify explicit fields in MANIFEST.MF in a jvm_binary target. RB #2199 RB #2084 RB #2119 RB #2005

• Parallelize isolated jvm compile strategy’s chunk execution. RB #2109

• Make test tasks specify which target failed in exception. RB #2090 RB #2113 RB #2112

• Support glob output in filedeps. RB #2092

• Export: support export of sources and globs RB #2082 RB #2094

• Classpath isolation: make ivy resolution locally accurate. RB #2064

• Add support for a postscript to jar_publish commit messages. RB #2070

• Add optional support for auto-shading jvm tools. RB #2052 RB #2073

• Introduce a jvm binary shader. RB #2050

• Open source the spindle plugin for pants into contrib. RB #2306 RB #2301 RB #2304 RB #2282 RB #2033

• Implement an exported ownership model. RB #2010

Small improvements, Refactoring and Tooling

• Support caching chroots for reuse across pants runs. RB #2349

• Upgrade RBT to the latest release RB #2360

• Make sure arg to logRaw and log are only eval’ed once. (zinc) RB #2338

• Clean up unnecessary code RB #2339

• Exclude the com.example org from travis ivy cache. RB #2344

• Avoid ivy cache thrash due to ivydata updates. RB #2333

• Various refactoring of PythonChroot and related code. RB #2327

• Have pytest_run create its chroots via its base class. RB #2314

• Add a set of memoization decorators for functions. RB #2308 RB #2317

• Allow jvm tool tests to bootstrap from the artifact cache. RB #2311

• Fixed ‘has no attribute’ exception + better tests for export goal RB #2305

• Refactoring ProtobufGen to use SimpleCodeGen. RB #2302

• Refactoring JaxbGen to use SimpleCodeGen. RB #2303

• Add pants header to assorted python files RB #2298

• Remove unused imports from python files RB #2295

• Integrating Patrick’s SimpleCodegenTask base class with WireGen. RB #2274

• Fix bad log statement in junit_run.py. RB #2290

• Provide more specific value parsing errors RB #2283

• Dry up incremental-compiler dep on sbt-interface. RB #2279

• Use BufferedOutputStream in jar-tool RB #2270

• Add relative_symlink to dirutil for latest run report RB #2271

• Shade zinc. RB #2268

• rm Exception.message calls RB #2245

• sanity check on generated cobertura xml report RB #2231

• [pants/jar] Fix a typo RB #2230

• Convert validation.assert_list isinstance checking to be lazy RB #2228

• use workunit output for cpp command running RB #2223

• Remove all global config state. RB #2222 RB #2181 RB #2160 RB #2159 RB #2151 RB #2142 RB #2141

• Make the version of specs in BUILD.tools match the one in 3rdparty/BUILD. RB #2203

• Handle warnings in BUILD file context. RB #2198

• Replace custom softreference cache with a guava cache. (zinc) RB #2190

• Establish a source_root for pants scala code. RB #2189

• Zinc patches to improve roundtrip time RB #2178

• cache parsed mustache templates as they are requested RB #2171

• memoize linkify to reduce reporting file stat calls RB #2170

• Refactor BuildFile and BuildFileAdressMapper RB #2110

• fix whitespace in workerpool test, rm unused import RB #2144

• Use jvm-compilers as the parent of isolation workunits instead of ‘isolation’, add workunits for analysis RB #2134

• Improve the error message when a tool fails to bootstrap. RB #2135

• Fix rglobs-to-filespec code. RB #2133

• Send workunit output to stderr during tests RB #2108

• Changes to zinc analysis split/merge test data generation: RB #2095

• Add a dummy workunit to the end of the run to print out a timestamp that includes the time spent in the last task. RB #2054

• Add ‘java-resource’ and ‘java-test-resource’ content type for Resources Roots. RB #2046

• Upgrade virtualenv from 12.0.7 to 12.1.1. RB #2047

• convert all % formatted strings under src/ to str.format format RB #2042

• Move overrides for registrations to debug. RB #2023

• Split jvm_binary.py into jvm_binary.py and jvm_app.py. RB #2006

• Validate analysis earlier, and handle it explicitly RB #1999

• Switch to importlib RB #2003

• Some refactoring and tidying-up in workunit. RB #1981

• Remove virtualenv tarball from CI cache. RB #2281

• Moved testing of examples and testprojects to tests RB #2158

• Share the python interpreter/egg caches between tests. RB #2256

• Add support for python test sharding. RB #2243

• Fixup OSX CI breaks. RB #2241

• fix test class name c&p error RB #2227

• Remove the pytest skip tag for scala publish integration test as it uses –doc-scaladoc-skip RB #2225

• integration test for classifiers RB #2216 RB #2218 RB #2232

• Use 2 IT shards to avoid OSX CI timeouts. RB #2217

• Don’t have JvmToolTaskTestBase require access to “real” option values. RB #2213

• There were two test_export_integration.py tests. RB #2215

• Do not include integration tests in non-integration tests. RB #2173

• Streamline some test setup. RB #2167

• Ensure that certain test cleanup always happens, even if setUp fails. RB #2166

• Added a test of the bootstrapper logic with no cached bootstrap.jar RB #2126

• Remove integration tests from default targets in test BUILD files RB #2086

• Cap BootstrapJvmTools mem in JvmToolTaskTestBase. RB #2077

• Re-establish no nailguns under TravisCI. RB #1852 RB #2065

• Further cleanup of test context setup. RB #2053

• Remove plumbing for custom test config. RB #2051

• Use a fake context when testing. RB #2049

• Remove old TaskTest base class. RB #2039 RB #2031 RB #2027 RB #2022 RB #2017 RB #2016

• Refactor com.pants package to org.pantsbuild in examples and testprojects RB #2037

• Added a simple ‘HelloWorld’ java example. RB #2028

• Place the workdir below the pants_workdir RB #2007

Bugfixes

• Fixup minified_dependencies Issue #1329 RB #1986

• Don`t mutate options in the linter RB #1978

• Fix a bad logic bug in zinc analysis split code RB #1969

• always use relpath on –test file args RB #1976

• Fixup resources drift in the sdist package RB #1974

• Fix publish override flag Issue #1277 RB #1959

API Changes

• Remove open_zip64 in favor of supporting zip64 everywhere RB #1984

Documentation

• rm python_old, an old document RB #1973

• Updated ivysettings.xml with comments and commented out local repos RB #1979

• Update how to setup proxies in ivy RB #1975

New Features

• Ignore blank lines and comments in scalastyle excludes file RB #1971

• Adding a –test-junit-coverage-jvm-options flag RB #1968

• –soft-excludes flag for resolve-ivy RB #1961

Small improvements, Refactoring and Tooling

• Rid pantsbuild.pants of an un-needed antlr dep RB #1989

• Kill the BUILD.transitional targets Issue #1126 RB #1983

• Convert ragel-gen.py to use new options and expunge config from BinaryUtil RB #1970

• Add the JvmCompileIsolatedStrategy RB #1898

• Move construction of PythonChroot to PythonTask base class RB #1965

• Delete the PythonBinaryBuilder class RB #1964

• Removing dead code RB #1960

• Make the test check that the return code is propagated RB #1966

• Cleanup RB #1962

• Get rid of almost all direct config access in python-building code RB #1954

Bugfixes

• Make JavaProtobufLibrary not exportable to fix publish. RB #1952

• Pass compression option along to temp local artifact caches. RB #1955

• Fix a missing symbol in ScalaCompile RB #1885 RB #1945

• die only when invoked directly RB #1953

• add import for traceback, and add test to exercise that code path, rm unsed kwargs RB #1868 RB #1943

API Changes

• Use the publically released 2.1.1 version of Cobertura RB #1933

Documentation

• Update docs for ‘prep_command()’ RB #1940

New Features

• added sources and javadocs to export goal output RB #1936

• Add flags to idea and eclipse goals to exclude pulling in sources and javadoc via ivy RB #1939

Small improvements, Refactoring and Tooling

• Remove a spurious import in test_antlr_builder RB #1951

• Refactor ZincUtils RB #1946

• change set([]) / OrderedSet([]) to set() / OrderedSet() RB #1947

• Rename TestPythonSetup to TestSetupPy RB #1950

• Rename the PythonSetup task to SetupPy RB #1942

Bugfixes

• Fix missing deps from global switch to six range RB #1931 RB #1937

• Fix python_repl to work for python_requirement_libraries RB #1934

• Move count variable outside loop RB #1926

• Fix regression in synthetic target context handling RB #1921

• Try to fix the .rst render of the CHANGELOG on pypi RB #1911

• To add android.jar to the classpath, create a copy under task’s workdir RB #1902

• walk synthetic targets dependencies when constructing context.target() RB #1863 RB #1914

• Mix the value of the zinc name-hashing flag into cache keys RB #1912

• Allow multiple ivy artifacts distinguished only by classifier RB #1905

• Fix Git.detect_worktree to fail gracefully RB #1903

• Avoid reparsing analysis repeatedly RB #, RB #1938

API Changes

• Remove the now-superfluous “parallel resource directories” hack RB #1907

• Make rglobs follow symlinked directories by default RB #1881

Documentation

• Trying to clarify how to contribute docs RB #1922

• Add documentation on how to turn on extra ivy debugging RB #1906

• Adds documentation to setup_repo.md with tips for how to configure Pants to work behind a firewall RB #1899

New Features

• Support spec_excludes in what_changed. Prior art: https://rbcommons.com/s/twitter/r/1795/ RB #1930

• Add a new ‘export’ goal for use by IDE integration RB #1917 RB #1929

• Add ability to detect HTTP_PROXY or HTTPS_PROXY in environment and pass it along to ivy RB #1877

• Pants publish to support publishing extra publish artifacts as individual artifacts with classifier attached RB #1879 RB #1889

Small improvements, Refactoring and Tooling

• Deleting dead abbreviate_target_ids code. RB #1918 RB #1944

• Move AptCompile to its own file RB #1935

• use six.moves.range everywhere RB #1931

• Port scrooge/linter config to the options system RB #1927

• Fixes for import issues in JvmCompileStrategy post https://rbcommons.com/s/twitter/r/1885/ RB #1900

• Moving stuff out of jvm and into project info backend RB #1917

• Provides is meant to have been deprecated a long time ago RB #1915

• Move JVM debug config functionality to the new options system RB #1924

• Remove the –color option from specs_run. See https://rbcommons.com/s/twitter/r/1814/ RB #1916

• Remove superfluous ‘self.conf’ argument to self.classpath RB #1913

• Update ivy_utils error messages: include classifier and switch interpolation from % to format RB #1908

• Added a python helper for check_header.sh in git pre-commit script RB #1910

• Remove direct config access in scalastyle.py RB #1897

• Replace all instances of xrange with range, as xrange is deprecated in Python 3 RB #1901

• Raise a better exception on truncated Zinc analysis files RB #1896

• Fail fast for OSX CI runs RB #1894

• Upgrade to the latest rbt release RB #1893

• Use cmp instead of a file hash RB #1892

• Split out a JvmCompileStrategy interface RB #1885

• Decouple WorkUnit from RunTracker RB #1928

• Add Scm.add, change publish to add pushdb explicitly, move scm publish around RB #1868

CI

• Support local pre-commit checks RB #1883

• Fix newline to fix broken master build RB #1888

• Shard out OSX CI RB #1873

• Update travis’s pants cache settings RB #1875

• Fixup contrib tests on osx CI RB #1867

• Reduce number of test shards from 8 to 6 on Travis-ci RB #1804

• Cache the isort venv for ci runs RB #1740

• Fixup ci isort check RB #1728

Tests

• Add jar Publish integration tests to test the generated pom and ivy.xml files RB #1879

• Added test that shows that nested scope inherits properly from cmdline, config, and env RB #1851 RB #1865

• Improve AndroidDistribution coverage RB #1861

• Modernize the protobuf and wire task tests RB #1854

• Replace python_test_suite with target RB #1821

• Switch test_jvm_run.py to the new TaskTestBase instead of the old TaskTest RB #1829

• Remove two non-useful tests RB #1828

• Fix a python run integration test RB #1810

• Work around py test_runner issue with ns packages RB #1813

• Add a test for the Git changelog RB #1792

• Create a directory with no write perms for TestAndroidConfigUtil RB #1796

• Relocated some tests (no code changes) from tests/python/pants_test/tasks into tests/python/pants_test/backend/codegen/tasks to mirror the source location RB #1746

Docs

• Add some documentation about using the pants reporting server for troubleshooting RB #1887

• Docstring reformatting for Task and InvalidationCheck RB #1769

• docs: Show correct pictures for intellij.html RB #1716

• doc += how to turn on cache RB #1668

New language: C++

• Separate compile step for C++ to just compile objects RB #1855

• Fixup CppToolchain to be lazy and actually cache RB #1850

• C++ support in contrib RB #1818

API Changes

• Kill the global –ng-daemons flag RB #1852

• Removed parallel_test_paths setting from pants.ini. It isn’t needed in the pants repo any more RB #1846

• BUILD file format cleanup:

  • Deprecate bundle().add in favor of bundle(files=) RB #1788

  • Deprecate .intransitive() in favor of argument RB #1797

  • Deprecate target.with_description in favor of target(description=) RB #1790

  • Allow exclude in globs RB #1762

  • Move with_artifacts to an artifacts argument RB #1672

• An attempt to deprecate some old methods RB #1720

• Options refactor work

  • Make option registration recursion optional RB #1870

  • Remove all direct config uses from jar_publish.py RB #1844

  • Read pants_distdir from options instead of config RB #1842

  • Remove direct config references in thrift gen code RB #1839

  • Android backend now exclusively uses the new option system RB #1819

  • Replace config use in RunTracker with options RB #1823

  • Add pants_bootstradir and pants_configdir to options bootstrapper RB #1835

  • Remove all direct config access in task.py RB #1827

  • Convert config-only options in goal idea and eclipse to use new options format RB #1805

  • Remove config_section from some tasks RB #1806

  • Disallow –no- on the name of boolean flags, refactor existing ones Issue #34 RB #1799

  • Migrating pants.ini config values for protobuf-gen to advanced registered options under gen.protobuf RB #1741

• Add a way to deprecate options with ‘deprecated_version’ and ‘deprecated_hint’ kwargs to register() RB #1799 RB #1814

• Implement compile_classpath using UnionProducts RB #1761

• Introduce a @deprecated decorator RB #1725

• Update jar-tool to 0.1.9 and switch to use @argfile calling convention RB #1798

• Pants to respect XDB spec for global storage on unix systems RB #1817

• Adds a mixin (ImportJarsMixin) for the IvyImports task RB #1783

• Added invalidation check to UnpackJars task RB #1776

• Enable python-eval for pants source code RB #1773

• adding xml output for python coverage Issue #1105 RB #1770

• Optionally adds a path value onto protoc’s PATH befor launching it RB #1756

• Add progress information to partition reporting RB #1749

• Add SignApk product and Zipalign task RB #1737

• Add an ‘advanced’ parameter to registering options RB #1739

• Add an env var for enabling the profiler RB #1305

Bugfixes and features

• Kill the .saplings split RB #1886

• Update our requests library to something more recent RB #1884

• Make a nicer looking name for workunit output RB #1876

• Fixup DxCompile jvm_options to be a list RB #1878

• Make sure <?xml starts at the beginning of the file when creating an empty xml report RB #1856

• Set print_exception_stacktrace in pants.ini RB #1872

• Handle –print-exception-stacktrace and –version more elegantly RB #1871

• Improve AndroidDistribution caching RB #1861

• Add zinc to the platform_tools for zinc_utils RB #1779 RB #1858

• Fix WARN/WARNING confusion RB #1866

• Fixup Config to find DEFAULT values for missing sections RB #1851

• Get published artifact classfier from config RB #1857

• Make Context.targets() include synthetic targets RB #1840 RB #1863

• Fix micros to be left 0 padded to 6 digits RB #1849

• Setup logging before plugins are loaded RB #1820

• Introduce pants_setup_py and contrib_setup_py helpers RB #1822

• Support zinc name hashing RB #1779

• Actually generate a depfile from t.c.tools.compiler and use it in jmake RB #1824 RB #1825

• Ivy Imports now has a cache RB #1785

• Get rid of some direct config uses in python_repl.py RB #1826

• Add check if jars exists before registering products RB #1808

• shlex the python run args RB #1782

• Convert t.c.log usages to logging RB #1815

• Kill unused twitter.common reqs and deps RB #1816

• Check import sorting before checking headers RB #1812

• Fixup typo accessing debug_port option RB #1811

• Allow the dependees goal and idea to respect the –spec_excludes option RB #1795

• Copy t.c.lang.{AbstractClass,Singleton} to pants RB #1803

• Replace all t.c.lang.Compatibility uses with six RB #1801

• Fix sp in java example readme.md RB #1800

• Add util.XmlParser and AndroidManifestParser RB #1757

• Replace Compatibility.exec_function with six.exec_ RB #1742 RB #1794

• Take care of stale pidfiles for pants server RB #1791

• Fixup the scrooge release RB #1793

• Extract scrooge tasks to contrib/ RB #1780

• Fixup JarPublish changelog rendering RB #1787

• Preserve dictionary order in the anonymizer RB #1779 RB #1781

• Fix a test file leak to the build root RB #1771

• Replace all instances of compatibility.string RB #1764

• Improve the python run error message RB #1773 RB #1777

• Upgrade pex to 0.8.6 RB #1778

• Introduce a PythonEval task RB #1772

• Add an elapsed timestamp to the banner for CI RB #1775

• Trying to clean up a TODO in IvyTaskMixin RB #1753

• rm double_dag RB #1711

• Add skip / target invalidation to thrift linting RB #1755

• Fixup Task.invalidated UI RB #1758

• Improve the implementation of help printing RB #1739 RB #1744

• Fix TestAndroidBase task_type override miss RB #1751

• Pass the BUILD file path to compile RB #1742

• Bandaid leaks of global Config state in tests RB #1750

• Fixing cobertura coverage so that it actually works RB #1704

• Restore the ability to bootstrap Ivy with a custom configuration file RB #1709

• Kill BUILD file bytecode compilation RB #1736

• Kill ‘goal’ usage in the pants script RB #1738

• Fixup ivy report generation and opening RB #1735

• Fixup pants sys.excepthook for pex context RB #1733 RB #1734

• Adding long form of help arguments to the help output RB #1732

• Simplify isort config RB #1731

• Expand scope of python file format checks RB #1729

• Add path-to option to depmap RB #1545

• Fix a stragler .is_apt usage RB #1724

• Introduce isort to check *.py import ordering RB #1726

• Upgrade to pex 0.8.5 RB #1721

• cleanup is_xxx checks: is_jar_library RB #1719

• Avoid redundant traversal in classpath calculation RB #1714

• Upgrade to the latest virtualenv RB #1715 RB #1718

• Fixup the release script RB #1715

• ‘./pants goal’ -> ‘./pants’ RB #1617

• Add new function open_zip64 which defaults allowZip64=True for Zip files RB #1708

• Fix a bug that –bundle-archive=tar generates .tar.gz instead of a .tar RB #1707

• Remove 3rdparty debug.keystore RB #1703

• Keystore no longer a target, apks signed with SignApkTask RB #1690

• remove this jar_rule I accidentally added RB #1701

• Require pushdb migration to specify a destination directory RB #1684

Bugfixes

• Numerous doc improvements & generation fixes

  • Steal some info from options docstring

  • Document –config-override & PANTS_ environment vars

  • Document JDK_HOME & JAVA_HOME use when choosing a java distribution

  • Rename “Goals Reference” page -> “Options Reference”

  • Document when to use isrequired

  • Fix Google indexing to ignore test sites

  • Update the code layout section of Pants Internals

  • Show changelog & for that support page(source=’something.rst’)

  • Add a reminder that you can do set-like math on FileSets

  • Hacking on Pants itself, update –pdb doc

  • Start of a “Why Choose Pants?” section

  • Highlight plugin examples from twitter/commons

  • Add a blurb about deploy_jar_rules to the JVM docs

  • Show how to pass -s to pytest

  • When to use java_sources, when not to

  • Start of a Pants-with-scala page

  • Publish page now shows provides= example

  • Add a flag to omit “internal” things

  • Slide tweaks based on class feedback

  • Document argument splitting for options

  Issue #897 RB #1092 RB #1490 RB #1532 RB #1544 RB #1546 RB #1548 RB #1549 RB #1550 RB #1554 RB #1555 RB #1559 RB #1560 RB #1565 RB #1575 RB #1580 RB #1583 RB #1584 RB #1593 RB #1607 RB #1608 RB #1609 RB #1618 RB #1622 RB #1633 RB #1640 RB #1657 RB #1658 RB #1563 RB #1564 RB #1677 RB #1678 RB #1694 RB #1695

• Add calls to relpath so that we don’t generate overlong filenames on mesos RB #1528 RB #1612 RB #1644

• Regularize headers RB #1691

• Pants itself uses python2.7, kill unittest2 imports RB #1689

• Make ‘setup-py’ show up in ‘./pants goal goals’ RB #1466

• Test that CycleException happens for cycles (instead of a stack overflow) RB #1686

• Replace t.c.collection.OrderedDict with 2.7+ stdlib RB #1687

• Make ide_gen a subclass of Task to avoid depending on compile and resources tasks Issue #997 RB #1679

• Remove with_sources() from 3rdparty/BUILD RB #1674

• Handle thrift inclusion for python in apache_thrift_gen RB #1656 RB #1675

• Make beautifulsoup4 dep fixed rather than floating RB #1670

• Fixes for unpacked_jars RB #1624

• Fix spurious Products requirements RB #1662

• Fixup the options bootstrapper to support boolean flags RB #1660 RB #1664

• Change Distribution.cached to compare using Revision objects RB #1653

• Map linux i686 arch to i386 Issue #962 RB #1659

• bump virtualenv version to 12.0.5 RB #1621

• Bugfixes in calling super methods in traversable_specs and traversable_dependency_specs RB #1611

• Raise TaskError on python antlr generation failure RB #1604

• Fix topological ordering + chunking bug in jvm_compile RB #1598

• Fix CI from RB 1604 (and change a test name as suggested by nhoward) RB #1606

• Mark some missing-deps testprojects as expected to fail RB #1601

• Fix scalac plugin support broken in a refactor RB #1596

• Do not insert an error message as the “main” class in jvm_binary_task RB #1590

• Remove variable shadowing from method in archive.py RB #1589

• Don’t realpath jars on the classpath RB #1588 RB #1591

• Cache ivy report dependency traversals consistently RB #1557

• Print the traceback when there is a problem loading or calling a backend module RB #1582

• Kill unused Engine.execution_order method and test RB #1576

• Support use of pytest’s –pdb mode RB #1570

• fix missing dep. allows running this test on its own RB #1561

• Remove dead code and no longer needed topo sort from cache_manager RB #1553

• Use Travis CIs new container based builds and caching RB #1523 RB #1537 RB #1538

API Changes

• Improvements and extensions of WhatChanged functionality

  • Skip loading graph if no changed targets

  • Filter targets from changed using exclude_target_regexp

  • Compile/Test “changed” targets

  • Optionally include direct or transitive dependees of changed targets

  • Add changes-in-diffspec option to what-changed

  • Refactor WhatChanged into base class, use LazySourceMapper

  • Introduce LazySourceMapper and test

  RB #1526 RB #1534 RB #1535 RB #1542 RB #1543 RB #1567 RB #1572 RB #1595 RB #1600

• More options migration, improvements and bugfixes

  • Centralize invertible arg logic

  • Support loading boolean flags from pants.ini

  • Add a clarifying note in migrate_config

  • Some refactoring of IvyUtils

  • Rename the few remaining “jvm_args” variables to “jvm_options”

  • ./pants –help-all lists all options

  • Add missing stanza in the migration script

  • Switch artifact cache setup from config to new options

  • Migrate jvm_compile’s direct config accesses to the options system

  • Added some formatting to parse errors for dicts and lists in options

  • s/new_options/options/g

  • Re-implement the jvm tool registration mechanism via the options system

  • Make JvmRun support passthru args

  RB #1347 RB #1495 RB #1521 RB #1527 RB #1552 RB #1569 RB #1585 RB #1599 RB #1626 RB #1630 RB #1631 RB #1646 RB #1680 RB #1681 RB #1696

• Upgrade pex dependency to 0.8.4

  • Pick up several perf wins

  • Pick up fix that allows pex to read older pexes

  RB #1648 RB #1693

• Upgrade jmake to org.pantsbuild releases

  • Upgrade jmake to version with isPackagePrivateClass fix

  • Upgrade jmake to version that works with java 1.5+

  Issue #13 RB #1594 RB #1628 RB #1650

• Fix ivy resolve args + added ability to provide custom ivy configurations RB #1671

• Allow target specs to come from files RB #1669

• Remove obsolete twitter-specific hack ‘is_classpath_artifact’ RB #1676

• Improve RoundEngine lifecycle RB #1665

• Changed Scala version from 2.9.3 to 2.10.3 because zinc was using 2.10.3 already RB #1610

• Prevent “round trip” dependencies RB #1603

• Edit Config.get_required so as to raise error for any blank options RB #1638

• Don’t plumb an executor through when bootstrapping tools RB #1634

• Print jar_dependency deprecations to stderr RB #1632

• Add configuration parameter to control the requirements cache ttl RB #1627

• Got ivy to map in javadoc and source jars for pants goal idea RB #1613 RB #1639

• Remove the ‘^’ syntax for the command line spec parsing RB #1616

• Kill leftover imports handling from early efforts RB #592 RB #1614

• Adding the ability to pull in a Maven artifact and extract its contents RB #1210

• Allow FingerprintStrategy to opt out of fingerprinting RB #1602

• Remove the ivy_home property from context RB #1592

• Refactor setting of PYTHONPATH in pants.ini RB #1586

• Relocate ‘to_jar_dependencies’ method back to jar_library RB #1574

• Update protobuf_gen to be able to reference sources outside of the subdirectory of the BUILD file RB #1573

• Kill goal dependencies RB #1577

• Move excludes logic into cmd_line_spec_parser so it can filter out broken build targets RB #930 RB #1566

• Replace exclusives_groups with a compile_classpath product RB #1539

• Allow adding to pythonpath via pant.ini RB #1457

API Changes

• Flesh out and convert to the new options system introduced in pantsbuild.pants 0.0.24

  • Support loading config from multiple files

  • Support option reads via indexing

  • Add a migrate_config tool

  • Migrate tasks to the option registration system

  • Get rid of the old config registration mechanism

  • Add passthru arg support in the new options system

  • Support passthru args in tasks

  • Allow a task type know its own options scope

  • Support old-style flags even in the new flag system

  RB #1093 RB #1094 RB #1095 RB #1096 RB #1097 RB #1102 RB #1109 RB #1114 RB #1124 RB #1125 RB #1127 RB #1129 RB #1131 RB #1135 RB #1138 RB #1140 RB #1146 RB #1147 RB #1170 RB #1175 RB #1183 RB #1186 RB #1192 RB #1195 RB #1203 RB #1211 RB #1212 RB #1214 RB #1218 RB #1223 RB #1225 RB #1229 RB #1230 RB #1231 RB #1232 RB #1234 RB #1236 RB #1244 RB #1248 RB #1251 RB #1258 RB #1269 RB #1270 RB #1276 RB #1281 RB #1286 RB #1289 RB #1297 RB #1300 RB #1308 RB #1309 RB #1317 RB #1320 RB #1323 RB #1328 RB #1341 RB #1343 RB #1351 RB #1357 RB #1373 RB #1375 RB #1385 RB #1389 RB #1399 RB #1409 RB #1435 RB #1441 RB #1442 RB #1443 RB #1451

• Kill Commands and move all actions to Tasks in the goal infrastructure

  • Kill pants own use of the deprecated goal command

  • Restore the deprecation warning for specifying ‘goal’ on the cmdline

  • Get rid of the Command class completely

  • Enable passthru args for python run

  RB #1321 RB #1327 RB #1394 RB #1402 RB #1448 RB #1453 RB #1465 RB #1471 RB #1476 RB #1479

• Add support for loading plugins via standard the pkg_resources entry points mechanism RB #1429 RB #1444

• Many performance improvements and bugfixes to the artifact caching subsystem

  • Use a requests Session to enable connection pooling

  • Make CacheKey hash and pickle friendly

  • Multiprocessing Cache Check and Write

  • Skip compressing/writing artifacts that are already in the cache

  • Add the ability for JVM targets to refuse to allow themselves to be cached in the artifact cache

  • Fix name of non-fatal cache exception

  • Fix the issue of seeing “Error while writing to artifact cache: an integer is required” during [cache check]

  • Fix all uncompressed artifacts stored as just .tar

  RB #981 RB #986 RB #1022 RB #1197 RB #1206 RB #1233 RB #1261 RB #1264 RB #1265 RB #1272 RB #1274 RB #1249 RB #1310

• More enhancements to the depmap goal to support IDE plugins:

  • Add Pants Target Type to depmap to identify scala target VS java target

  • Add java_sources to the depmap info

  • Add transitive jar dependencies to depmap project info goal for intellij plugin

  RB #1366 RB #1324 RB #1047

• Port pants to pex 0.8.x Issue #10 Issue #19 Issue #21 Issue #22 RB #778 RB #785 RB #1303 RB #1378 RB #1421

• Remove support for __file__ in BUILDs RB #1419

• Allow setting the cwd for goals run.jvm and test.junit RB #1344

• Subclasses of Exception have strange deserialization RB #1395

• Remove outer (pants_exe) lock and serialized cmd RB #1388

• Make all access to Context’s lock via helpers RB #1391

• Allow adding entries to source_roots RB #1359

• Re-upload artifacts that encountered read-errors RB #1361

• Cache files created by (specially designed) annotation processors RB #1250

• Turn dependency dupes into errors RB #1332

• Add support for the Wire protobuf library RB #1275

• Pin pants support down to python2.7 - dropping 2.6 RB #1278

• Add a new param for page target, links, a list of hyperlinked-to targets RB #1242

• Add git root calculation for idea goal RB #1189

• Minimal target “tags” support RB #1227

• Include traceback with failures (even without fail-fast) RB #1226

• Add support for updating the environment from prep_commands RB #1222

• Read arguments for thrift-linter from pants.ini RB #1215

• Configurable Compression Level for Cache Artifacts RB #1194

• Add a flexible directory re-mapper for the bundle RB #1181

• Adds the ability to pass a filter method for ZIP extraction RB #1199

• Print a diagnostic if a BUILD file references a source file that does not exist RB #1198

• Add support for running a command before tests RB #1179 RB #1177

• Add PantsRunIntegrationTest into pantsbuild.pants.testinfra package RB #1185

• Refactor jar_library to be able to unwrap its list of jar_dependency objects RB #1165

• When resolving a tool dep, report back the pants.ini section with a reference that is failing RB #1162

• Add a list assertion for python_requirement_library’s requirements RB #1142

• Adding a list of dirs to exclude from the ‘::’ scan in the CmdLineSpecParser RB #1091

• Protobuf and payload cleanups RB #1099

• Coalesce errors when parsing BUILDS in a spec RB #1061

• Refactor Payload RB #1063

• Add support for publishing plugins to pants RB #1021

Bugfixes

• Numerous doc improvements & generation fixes

  • Updates to the pants essentials tech talk based on another dry-run

  • On skinny displays, don’t show navigation UI by default

  • Handy rbt status tip from RBCommons newsletter

  • Document how to create a simple plugin

  • Update many bash examples that used old-style flags

  • Update Pants+IntelliJ docs to say the Plugin’s the new hotness, link to plugin’s README

  • Publish docs the new way

  • Update the “Pants Essentials” tech talk slides

  • Convert .rst files -> .md files

  • For included code snippets, don’t just slap in a pre, provide syntax highlighting

  • Add notes about JDK versions supported

  • Dust off the Task Developer’s Guide and rm the “pagerank” example

  • Add a sitegen task, create site with better navigation

  • For ‘goal builddict’, generate .rst and .html, not just .rst

  • Narrow setup ‘Operating System’ classfiers to known-good

  Issue #16 Issue #461 Issue #739 RB #891 RB #1074 RB #1075 RB #1079 RB #1084 RB #1086 RB #1088 RB #1090 RB #1101 RB #1126 RB #1128 RB #1134 RB #1136 RB #1154 RB #1155 RB #1164 RB #1166 RB #1176 RB #1178 RB #1182 RB #1191 RB #1196 RB #1205 RB #1241 RB #1263 RB #1277 RB #1284 RB #1292 RB #1295 RB #1296 RB #1298 RB #1299 RB #1301 RB #1314 RB #1315 RB #1326 RB #1348 RB #1355 RB #1356 RB #1358 RB #1363 RB #1370 RB #1377 RB #1386 RB #1387 RB #1401 RB #1407 RB #1427 RB #1430 RB #1434 RB #1440 RB #1446 RB #1464 RB #1484 RB #1491

• CmdLineProcessor uses binary class name RB #1489

• Use subscripting for looking up targets in resources_by_products RB #1380

• Fix/refactor checkstyle RB #1432

• Fix missing import RB #1483

• Make ./pants help and ./pants help <goal> work properly Issue #839 RB #1482

• Cleanup after custom options bootstrapping in reflect RB #1468

• Handle UTF-8 in thrift files for python RB #1459

• Optimize goal changed RB #1470

• Fix a bug where a request for help wasn’t detected RB #1467

• Always relativize the classpath where possible RB #1455

• Gracefully handle another run creating latest link RB #1396

• Properly detect existence of a symlink RB #1437

• Avoid throwing in ApacheThriftGen.__init__ RB #1428

• Fix error message in scrooge_gen RB #1426

• Fixup BuildGraph to handle mixes of synthetic and BUILD targets RB #1420

• Fix antlr package derivation RB #1410

• Exit workers on sigint rather than ignore RB #1405

• Fix error in string formatting RB #1416

• Add missing class RB #1414

• Add missing import for dedent in resource_mapping.py RB #1403

• Replace twitter commons dirutil Lock with lockfile wrapper RB #1390

• Make interpreter_cache a property, acquire lock in accessor Issue #819 RB #1392

• Fix .proto files with unicode characters in the comments RB #1330

• Make pants goal run for Python exit with error code 1 if the python program exits non-zero RB #1374

• Fix a bug related to adding sibling resource bases RB #1367

• Support for the –kill-nailguns option was inadvertently removed, this puts it back RB #1352

• fix string formatting so test -h does not crash RB #1353

• Fix java_sources missing dep detection RB #1336

• Fix a nasty bug when injecting target closures in BuildGraph RB #1337

• Switch src/* usages of Config.load to use Config.from_cache instead RB #1319

• Optimize what_changed, remove un-needed extra sort RB #1291

• Fix DetectDuplicate’s handling of an append-type flag RB #1282

• Deeper selection of internal targets during publishing RB #1213

• Correctly parse named_is_latest entries from the pushdb RB #1245

• Fix error message: add missing space RB #1266

• WikiArtifact instances also have provides; limit ivy to jvm RB #1259

• Fix [run.junit] -> [test.junit] RB #1256

• Fix signature in goal targets and BUILD dictionary RB #1253

• Fix the regression introduced in https://rbcommons.com/s/twitter/r/1186 RB #1254

• Temporarily change stderr log level to silence log.init if –quiet RB #1243

• Add the environment’s PYTHONPATH to sys.path when running dev pants RB #1237

• Remove java_sources as target roots for scala library in depmap project info Issue #670 RB #1190

• Allow UTF-8 characters in changelog RB #1228

• Ensure proper semantics when replacing all tasks in a goal RB #1220 RB #1221

• Fix reading of scalac plugin info from config RB #1217

• Dogfood bintray for pants support binaries RB #1208

• Do not crash on unicode filenames RB #1193 RB #1209

• In the event of an exception in jvmdoc_gen, call get() on the remaining futures RB #1202

• Move workdirs creation from __init__ to pre_execute in jvm_compile & Remove QuietTaskMixin from several tasks RB #1173

• Switch from os.rename to shutil.move to support cross-fs renames when needed RB #1157

• Fix scalastyle task, wire it up, make configs optional RB #1145

• Fix issue 668: make release.sh execute packaged pants without loading internal backends during testing Issue #668 RB #1158

• Add payload.get_field_value() to fix KeyError from pants goal idea testprojects:: RB #1150

• Remove debug_args from pants.ini Issue #650 RB #1137

• When a jvm doc tool (e.g. scaladoc) fails in combined mode, throw an exception RB #1116

• Remove hack to add java_sources in context RB #1130

• Memoize Address.__hash__ computation RB #1118

• Add missing coverage deps RB #1117

• get goal targets using similar codepath to goal builddict RB #1112

• Memoize fingerprints by the FPStrategy hash RB #1119

• Factor in the jvm version string into the nailgun executor fingerprint RB #1122

• Fix some error reporting issues RB #1113

• Retry on failed scm push; also, pull with rebase to increase the odds of success RB #1083

• Make sure that ‘option java_package’ always overrides ‘package’ in protobuf_gen RB #1108

• Fix order-dependent force handling: if a version is forced in one place, it is forced everywhere RB #1085

• Survive targets without derivations RB #1066

• Make internal_backend plugins 1st class local pants plugins RB #1073

API Changes

• Add a whitelist to jvm dependency analyzer RB #888

• Refactor exceptions in build_file.py and build_file_parser.py to derive from a common baseclass and eliminate throwing IOError. RB #954

• Support absolute paths on the command line when they start with the build root RB #867

• Make :: fail for an invalid dir much like : does for a dir with no BUILD file Issue #484 RB #907

• Deprecate pants & dependencies aliases and remove config, goal, phase, get_scm & set_scm aliases RB #899 RB #903 RB #912

• Export test infrastructure for plugin writers to use in pantsbuild.pants.testinfra sdist Issue #539 RB #997 RB #1004

• Publishing improvements:

  • Add support for doing remote publishes with an explicit snapshot name

  • One publish/push db file per artifact

  RB #923 RB #994

• Several improvements to IdeGen derived goals:

  • Adds the –<goal>-use-source-root for IDE project generation tasks

  • Added –idea-exclude-maven-target to keep IntelliJ from indexing ‘target’ directories

  • Changes the behavior of goal idea to create a subdirectory named for the project name

  • Added exclude-folders option in pants.ini, defaulted to excluding a few dirs in .pants.d

  Issue #564 RB #1006 RB #1017 RB #1019 RB #1023

• Enhancements to the depmap goal to support IDE plugins:

  • Add flag to dump project info output to file

  • Add missing resources to targets

  • Add content type to project Info

  Issue #5 RB #964 RB #987 RB #998

• Make SourceRoot fundamentally understand a rel_path RB #1036

• Added thrift-linter to pants RB #1044

• Support limiting coverage measurements globally by module or path Issue #328 Issue #369 RB #1034

• Update interpreter_cache.py to support a repo-wide interpreter requirement RB #1025

• Changed goal markdown:

  • Writes output to ./dist/markdown/

  • Pages can include snippets from source files http://pantsbuild.github.io/page.html#include-a-file-snippet

  Issue #535 RB #949 RB #961

• Rename Phase -> Goal RB #856 RB #879 RB #880 RB #887 RB #890 RB #910 RB #913 RB #915 RB #931

• Android support additions:

  • Add AaptBuild task

  • Add JarsignerTask and Keystore target

  RB #859 RB #883

• Git/Scm enhancements:

  • Allow the buildroot to be a subdirectory of the git worktree

  • Support getting the commit date of refs

  • Add merge-base and origin url properties to git

  Issue #405 RB #834 RB #871 RB #884 RB #886

Bugfixes

• Numerous doc improvements & generation fixes Issue #397 Issue #451 Issue #475 RB #863 RB #865 RB #873 RB #876 RB #885 RB #938 RB #953 RB #960 RB #965 RB #992 RB #995 RB #1007 RB #1008 RB #1018 RB #1020 RB #1048

• Fixup missing ‘page.mustache’ resource for markdown goal Issue #498 RB #918

• Publishing fixes:

  • Fix credentials fetching during publishing

  • Skipping a doc phase should result in transitive deps being skipped as well

  RB #901 RB #1011

• Several IdeGen derived task fixes:

  • Fix eclipse_gen & idea_gen for targets with both java and scala

  • Fixup EclipseGen resources globs to include prefs.

  • When a directory contains both java_library and junit_tests targets, make sure the IDE understands this is a test path, not a lib path

  RB #857 RB #916 RB #996

• Fixes to the depmap goal to support IDE plugins:

  • Fixed source roots in project info in case of ScalaLibrary with java_sources

  • Fix –depmap-project-info for scala sources with the same package_prefix

  • Fix depmap KeyError

  RB #955 RB #990 RB #1015

• Make a better error message when os.symlink fails during bundle RB #1037

• Faster source root operations - update the internal data structure to include a tree RB #1003

• The goal filter’s –filter-ancestor parameter works better now Issue #506 RB #925

• Fix: goal markdown failed to load page.mustache Issue #498 RB #918

• Fix the changed goal so it can be run in a repo with a directory called ‘build’ RB #872

• Patch JvmRun to accept JvmApp`s `RB #893

• Add python as default codegen product RB #894

• Fix the filedeps goal - it was using a now-gone .expand_files() API Issue #437, RB #939

• Put back error message that shows path to missing BUILD files RB #929

• Make sure the junit_run task only runs on targets that are junit compatible Issue #508 RB #924

• Fix ./pants goal targets Issue #333 RB #796 RB #914

• Add derived_from to ScroogeGen synthetic targets RB #926

• Properly order resources for pants goal test and pants goal run RB #845

• Fixup Dependencies to be mainly target-type agnostic Issue #499 RB #920

• Fixup JvmRun only-write-cmd-line flag to accept relative paths Issue #494 RB #908 RB #911

• Fix the –ivy-report option and add integration test RB #976

• Fix a regression in Emma/Cobertura and add tests Issue #508 RB #935

API Changes

• Remove unused Task.invalidate_for method and unused extra_data variable RB #849

• Add DxCompile task to android backend RB #840

• Change all Task subclass constructor args to (*args, **kwargs) RB #846

• The public API for the new options system Issue #425 RB #831 RB #819

• Rename pants.goal.goal.Goal to pants.goal.task_registrar.TaskRegistrar Issue #345 RB #843

Bugfixes

• Better validation for AndroidTarget manifest field RB #860

• Remove more references to /BUILD:target notation in docs RB #855 RB #853

• Fix up the error message when attempting to publish without any configured repos RB #850

• Miscellaneous fixes to protobuf codegen including handling collisions deterministically RB #720

• Migrate some reasonable default values from pants.ini into ‘defaults’ in the pants source Issue #455 Issue #456 Issue #458 RB #852

• Updated the basename and name of some targets to prevent colliding bundles in dist/ RB #847

• Provide a better error message when referencing the wrong path to a BUILD file RB #841

• Add assert_list to ensure an argument is a list - use this to better validate many targets RB #811

• Update front-facing help and error messages for Android targets/tasks RB #837

• Use JvmFingerprintStrategy in cache manager RB #835

API Changes

• Upgrade pex dependency from twitter.common.python 0.6.0 to pex 0.7.0 RB #825

• Added a –spec-exclude command line flag to exclude specs by regular expression RB #747

• Upgrade requests, flip to a ranged requirement to help plugins RB #771

• New goal ensime to generate Ensime projects for Emacs users RB #753

Bugfixes

• goal repl consumes targets transitively RB #781

• Fixup JvmCompile to always deliver non-None products that were required by downstream RB #794

• Relativize classpath for non-ng java execution RB #804

• Added some docs and a bugfix on debugging a JVM tool (like jar-tool or checkstyle) locally RB #791

• Added an excludes attribute that is set to an empty set for all SourcePayload subclasses Issue #414 RB #793

• Add binary fetching support for OSX 10.10 and populate thrift and protoc binaries RB #789

• Fix the pants script exit status when bootstrapping fails RB #779

• Added benchmark target to maven_layout() RB #780

• Fixup a hole in external dependency listing wrt encoding RB #776

• Force parsing for filtering specs RB #775

• Fix a scope bug for java agent manifest writing RB #768 RB #770

• Plumb ivysettings.xml location to the publish template RB #764

• Fix goal markdown: README.html pages clobbered each other RB #750

API Changes

• Hide stack traces by default Issue #326 RB #655

• Upgrade to twitter.common.python 0.6.0 and adjust to api change RB #746

• Add support for Cobertura coverage Issue #70 RB #637

• Validate that junit_tests targets have non-empty sources RB #619

• Add support for the Ragel state-machine generator Issue #353 RB #678

• Add AndroidTask and AaptGen tasks RB #672 RB #676 RB #700

Bugfixes

• Numerous doc fixes Issue #385 Issue #387 Issue #395 RB #728 RB #729 RB #730 RB #738

• Expose types needed to specify jvm_binary.deploy_jar_rules Issue #383 RB #727

• Require information about jars in depmap with --depmap-project-info RB #721

API Changes

• Enable Nailgun Per Task RB #687

Bugfixes

• Numerous doc fixes RB #699 RB #703 RB #704

• Fixup broken bundle alias Issue #375 RB #722

• Remove dependencies on twitter.common.{dirutil,contextutils} RB #710 RB #713 RB #717 RB #718 RB #719 RB #726

• Fixup missing JunitRun resources requirement RB #709

• Fix transitive dependencies for GroupIterator/GroupTask RB #706

• Ensure resources are prepared after compile Issue #373 RB #708

• Upgrade to twitter.common.python 0.5.10 to brings in the following bugfix:

  Update the mtime on retranslation of existing distributions.
  
  1bff97e stopped existing distributions from being overwritten, to
  prevent subtle errors. However without updating the mtime these
  distributions will appear to be permanently expired wrt the ttl.

  RB #707

• Resurrected pants goal idea with work remaining on source and javadoc jar mapping RB #695

• Fix BinaryUtil raise of BinaryNotFound Issue #367 RB #705

API Changes

• Lock globs into rootdir and below Issue #348 RB #686

Bugfixes

• Several doc fixes RB #654 RB #693

• Fix relativity of antlr sources RB #679