-
Notifications
You must be signed in to change notification settings - Fork 14
Document usage of BFG and modify usage to be more ergonomic #24
base: master
Are you sure you want to change the base?
Changes from all commits
659ee55
c9979ed
c31bbae
9bacb72
db74785
2fba201
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||
|---|---|---|---|---|
| @@ -1,21 +1,64 @@ | ||||
| [](https://ci.bazel.io/job/BUILD_file_generator) | ||||
|
|
||||
| # BUILD File Generator | ||||
| # BUILD File Generator (BFG) | ||||
|
|
||||
| BUILD File Generator generates Bazel BUILD files for Java code. | ||||
| BUILD File Generator, or BFG for short, is a tool for generating Bazel BUILD files from one's source code. Instead of manually going through your project and creating BUILD files and their appropriate build rules, you can rely on BFG to do it for you! | ||||
|
|
||||
| 1. It reads all `.java` files, and extracts the class dependency graph. | ||||
| 2. Computes the strongly connected components of the graph. | ||||
| 3. For each component, creates a `java_library` rule. | ||||
| An example project can be found here. TODO | ||||
|
|
||||
| ### Why is it useful? | ||||
|
|
||||
| Bazel promises fast and correct builds, _especially_ for incremental builds. Rather than rebuild your entire codebase, Bazel _only_ rebuilds what is necessary, the targets that you have changed. For the remainder of your code, it relies on cached versions. | ||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm super late to the party, so feel free to reject. I suggest the following as a replacement for this section - """ This project automates writing granular BUILD rules that allow Bazel to parallelize and cache builds. It's useful to quickly try out Bazel on your project, as well as to periodically optimize your build graph. |
||||
|
|
||||
| There is one caveat to this fast incremental performance, though. It is dependent on the granularity of your targets. At one extreme, the least granular end, you can define a single glob for your entire project. This would be the quickest way to get your project using Bazel. However, it comes at the cost of eradicating any of Bazel's incremental build performance. On the other end, you can try to manually define the most granular targets possible, a single rule per file, collapsing any cyclic dependencies into the same target. This would ensure you make use of Bazel's blazingly fast incremental builds. However, it is extremely tedious, and somewhat annoying to maintain. | ||||
|
|
||||
| BFG automates this difficult tango. | ||||
|
|
||||
| ## Usage | ||||
|
|
||||
| TODO | ||||
| BFG is composed of two general components | ||||
|
|
||||
| 1. Language specific parsers (LSP) | ||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. perhaps "parser" instead of "LSP" thorough the doc? |
||||
| 2. The BFG binary | ||||
|
|
||||
| The LSPs read your source code and generate a class dependency graph in the form of a protobuf. To generate your BUILD files, you pass the generated protobuf into the BFG binary. | ||||
|
|
||||
| ### Step 0: Installation Instructions | ||||
|
|
||||
| To install BFG...TODO | ||||
|
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @cgrushko shall I open an issue for "how to install bfg"?
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is it much more than cloning the repo and building two targets? |
||||
|
|
||||
| ### Step 1: Using LSPs to generate dependency graphs | ||||
|
|
||||
| TODO(bazel-devel): add explanation and valid example arguments. | ||||
|
|
||||
| ```bash | ||||
| bazel run //lang:JavaSourceFileParserCli <TODO> | ||||
|
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @cgrushko how do you actually run the I wasn't able to immediately figure out what the command line format was, or how the input proto should be formatted.
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. For google-java-format it should probably be It's a bit silly that we have to pass both roots and files, but it comes from Google where you don't want to read the entire java tree just because it's the root. We can perhaps improve this by scanning all of roots if the user doesn't provide source files, but it's for another day. It should print usage information if run without flags (but I haven't tried it lately). Does it now? |
||||
| ``` | ||||
|
|
||||
| This generates a protobuf at TODO. | ||||
| TODO(bazel-devel): add clear protocol for how the protos are structured. Useful for other developers who may want to write LSPs. | ||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. we can link to the proto definition,
|
||||
|
|
||||
| ### Step 2: Generating BUILD files using BFG binary | ||||
|
|
||||
| TODO(bazel-devel): add explanation and valid example arguments. | ||||
|
|
||||
| ```bash | ||||
| bazel run //src:bfg <TODO> | ||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. //src/main/java/com/google/devtools/build/bfg ? |
||||
| ``` | ||||
|
|
||||
| ### Supported Languages | ||||
|
|
||||
| We currently support Java projects. The next language on our roadmap is Scala. | ||||
|
|
||||
| ## Development | ||||
|
|
||||
| ### Contributing | ||||
|
|
||||
| We welcome contributions! We generally prefer contributions that either (1) add features by extending BFG to other languages, (2) fix existing bugs, or (3) present bugs through the use of example projects. | ||||
|
|
||||
| ## Adding or updating third-party dependencies | ||||
| ### Dependency Management | ||||
|
|
||||
| We use a third-party tool called [bazel-deps](https://github.com/johnynek/bazel-deps) | ||||
| to manage our dependencies. All of our dependencies are listed in | ||||
| To manage our dependencies, we use a third-party tool called [bazel-deps](https://github.com/johnynek/bazel-deps). All of our dependencies are listed in | ||||
| [`maven_deps.yaml`](maven_deps.yaml). bazel-deps provides tools to manage | ||||
| adding and updating dependencies in that file, and generates the Bazel build | ||||
| files for them in `thirdparty/jvm/`. | ||||
|
|
||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| licenses(["notice"]) # Apache 2.0 | ||
|
|
||
| java_binary( | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. do you prefer not using //lang/java/src/main/java/com/google/devtools/build/bfg:JavaSourceFileParserCli ? |
||
| name = "JavaSourceFileParserCli", | ||
| main_class = "com.google.devtools.build.bfg.JavaSourceFileParserCli", | ||
| runtime_deps = ["//lang/java/src/main/java/com/google/devtools/build/bfg:JavaSourceFileParser"], | ||
| ) | ||
| Original file line number | Diff line number | Diff line change | ||
|---|---|---|---|---|
| @@ -1 +1,16 @@ | ||||
| A directory for language-specific parsers. | ||||
| # Language Specific Parsers | ||||
|
|
||||
| This directory contains a set of language-specific parsers, that can be used to generate a class and file level dependency graph of your project. It outputs this dependency graph as a protobuff. For users of bfg, this dependency graph can be used to generate your BUILD files. | ||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. "that can be used to generate a class..." -> that are used to provide BFG with dependency information. "protobuff" - we can link to the definition,
"bfg" -> "BFG" ? |
||||
|
|
||||
| We currently only support Java. However, we plan to add Scala support, and welcome contributions for other languages. | ||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'd drop this sentence, it doesn't add information - the user can see which directories exist there |
||||
|
|
||||
| ### Usage | ||||
|
|
||||
| ```bash | ||||
|
|
||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. superfluous newlines? |
||||
| bazel run //lang:JavaSourceFileParserCli | ||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. do you prefer not using //lang/java/src/main/java/com/google/devtools/build/bfg:JavaSourceFileParserCli ? |
||||
|
|
||||
| ``` | ||||
|
|
||||
| TODO(bazel-devel): what do the command line options mean??? There is a TODO in the Cli asking how the files will be obtained. | ||||
|
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @cgrushko similar question as before. How do you actually run the
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. See above. I'm not sure what the TODO means (we should figure out, or delete it :) |
||||
|
|
||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -30,10 +30,6 @@ java_library( | |
| "//thirdparty/jvm/com/google/guava", | ||
| "//thirdparty/jvm/org/eclipse/jdt:org_eclipse_jdt_core", | ||
| ], | ||
| visibility = ["//lang:__subpackages__"], | ||
|
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @cgrushko is this the appropriate visibility for the For context, I moved the rather than the more tedious The previous approach was annoyingly tedious. As far as the visibility, I want to expose
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yeah, you can't set visibility to a single rule. How often do you expect users to build the thing? We can add an |
||
| ) | ||
|
|
||
| java_binary( | ||
| name = "JavaSourceFileParserCli", | ||
| main_class = "com.google.devtools.build.bfg.JavaSourceFileParserCli", | ||
| runtime_deps = [":JavaSourceFileParser"], | ||
| ) | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -36,8 +36,9 @@ | |
| import protos.com.google.devtools.build.bfg.Bfg; | ||
|
|
||
| /** | ||
| * Entry point to BFG java source file parser. Given a list of source files, it prints a graph viz | ||
| * dot file representing the class level dependencies between these source files. | ||
| * Entry point to the BFG java source file parser. Given a list of source files, | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. let's leave this to another PR? |
||
| * it obtains the class and file level dependencies between the files, generates a | ||
| * dependency graph, and outputs this graph as a protobuf. | ||
| */ | ||
| public class JavaSourceFileParserCli { | ||
|
|
||
|
|
@@ -73,7 +74,8 @@ private void run(String[] args) throws Exception { | |
| } catch (CmdLineException e) { | ||
| if (sourceFiles.isEmpty()) { | ||
| System.err.println("Must provide file names to parse."); | ||
| } else { | ||
| //TODO(bazel-team) print example usage of the class | ||
| } else { | ||
| System.err.println(e.getMessage()); | ||
| } | ||
| e.getParser().printUsage(System.err); | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| licenses(["notice"]) # Apache 2.0 | ||
|
|
||
| package(default_visibility = [ | ||
| "//visibility:private", | ||
| ]) | ||
|
|
||
| java_binary( | ||
| name = "bfg", | ||
| main_class = "com.google.devtools.build.bfg.Bfg", | ||
| stamp = 1, | ||
| runtime_deps = ["//src/main/java/com/google/devtools/build/bfg:Bfg"], | ||
| ) | ||
|
|
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -76,13 +76,9 @@ java_library( | |
| "//thirdparty/jvm/com/google/guava", | ||
| "//thirdparty/jvm/com/google/re2j", | ||
| ], | ||
| ) | ||
|
|
||
| java_binary( | ||
| name = "bfg", | ||
| main_class = "com.google.devtools.build.bfg.Bfg", | ||
| stamp = 1, | ||
| runtime_deps = [":Bfg"], | ||
| visibility = [ | ||
|
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @cgrushko similar visibility questions. |
||
| "//src:__subpackages__", | ||
| ], | ||
| ) | ||
|
|
||
| java_library( | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I feel more is less here :) does it really contribute? as an engineer I personally prefer as little prose as possible :)
In this respect, laying out in bullets what the project does (even if it simplifies things) makes it easier for me to quickly scan the readme and see if the project is right for me.