Integrate graphviz into Mill docs, sprinkle diagrams throughout the d…

…ocsite (#3295)

Many of these diagrams were already present in various blog posts, so I
just copied those over. Others I wrote from scratch, which using
graphviz isnt too hard

build.sc

@@ -1670,10 +1670,11 @@ object docs extends Module {
       commandArgs = Seq(
         npmExe,
         "install",
-        "@antora/cli@3.0.1",
-        "@antora/site-generator-default@3.0.1",
+        "@antora/cli@3.1.9",
+        "@antora/site-generator-default@3.1.9",
         "gitlab:antora/xref-validator",
-        "@antora/lunr-extension@v1.0.0-alpha.6"
+        "@antora/lunr-extension@v1.0.0-alpha.6",
+        "asciidoctor-kroki@0.18.1"
       ),
       envArgs = Map(),
       workingDir = npmDir
@@ -1791,7 +1792,9 @@ object docs extends Module {
        |    utest-github-url: https://github.com/com-lihaoyi/utest
        |    upickle-github-url: https://github.com/com-lihaoyi/upickle
        |    mill-scip-version: ${Deps.DocDeps.millScip.dep.version}
-       |
+       |    kroki-fetch-diagram: true
+       |  extensions:
+       |  - asciidoctor-kroki
        |antora:
        |  extensions:
        |  - require: '@antora/lunr-extension'

docs/modules/ROOT/pages/Mill_Design_Principles.adoc

@@ -10,6 +10,8 @@ Mill:
 * http://www.lihaoyi.com/post/SowhatswrongwithSBT.html[Blog Post: So, what's wrong with SBT?]
 * http://www.lihaoyi.com/post/BuildToolsasPureFunctionalPrograms.html[Blog Post: Build Tools as Pure Functional Programs]
 
+== Principles
+
 === Dependency graph first
 
 Mill's most important abstraction is the dependency graph of ``Task``s.
@@ -196,6 +198,22 @@ build-related questions listed above.
 
 === The Object Hierarchy
 
+[graphviz]
+....
+digraph G {
+  node [shape=box width=0 height=0 style=filled fillcolor=white]
+  bgcolor=transparent
+  "root-module" [style=dashed]
+  foo1 [style=dashed]
+  foo2 [style=dashed]
+  "root-module" -> foo1 -> "foo1.bar"  [style=dashed]
+  foo1 -> "foo1.qux"  [style=dashed]
+  "root-module" -> foo2 -> "foo2.bar"  [style=dashed]
+  foo2 -> "foo2.qux"  [style=dashed]
+  foo2 -> "foo2.baz"  [style=dashed]
+}
+....
+
 The module hierarchy is the graph of objects, starting from the root of the
 `build.sc` file, that extend `mill.Module`. At the leaves of the hierarchy are
 the ``Target``s you can run.
@@ -220,6 +238,44 @@ are sure that it will never clash with any other ``Target``s data.
 
 === The Call Graph
 
+[graphviz]
+....
+digraph G {
+  rankdir=LR
+  node [shape=box width=0 height=0 style=filled fillcolor=white]
+  bgcolor=transparent
+  newrank=true;
+  subgraph cluster_0 {
+    style=dashed
+    node [shape=box width=0 height=0 style=filled fillcolor=white]
+    label = "foo.bar";
+
+    "foo.bar.sources" -> "foo.bar.compile" -> "foo.bar.classPath" -> "foo.bar.assembly"
+    "foo.bar.mainClass" -> "foo.bar.assembly"
+  }
+  subgraph cluster_1 {
+    style=dashed
+    node [shape=box width=0 height=0 style=filled fillcolor=white]
+    label = "foo";
+
+    "foo.bar.classPath" -> "foo.compile"   [constraint=false];
+    "foo.bar.classPath" -> "foo.classPath"
+    "foo.sources" -> "foo.compile" -> "foo.classPath" -> "foo.assembly"
+    "foo.mainClass" -> "foo.assembly"
+  }
+  subgraph cluster_2 {
+    style=dashed
+    node [shape=box width=0 height=0 style=filled fillcolor=white]
+    label = "qux";
+
+    "qux.mainClass" -> "qux.assembly"
+    "foo.classPath" -> "qux.compile" [constraint=false];
+    "foo.classPath" -> "qux.classPath"
+    "qux.sources" -> "qux.compile" -> "qux.classPath" -> "qux.assembly"
+  }
+}
+....
+
 The Scala call graph of "which target references which other target" is core to
 how Mill operates. This graph is reified via the `T {...}` macro to make it
 available to the Mill execution engine at runtime. The call graph tells you:

example/basic/2-custom-build-logic/build.sc

@@ -24,6 +24,26 @@ object foo extends RootModule with ScalaModule {
 
 //// SNIPPET:END
 
+// The addition of `lineCount` and `resources` overrides the previous `resource`
+// folder provided by `JavaModule` (labelled `resource.super` below), replacing
+// it with the destination folder of the new `resources` target, which is wired
+// up `lineCount`:
+//
+// [graphviz]
+// ....
+// digraph G {
+//   rankdir=LR
+//   node [shape=box width=0 height=0 style=filled fillcolor=white]
+//   src -> allSourceFiles -> lineCount -> resources -> dest -> run
+//   "resources.super" -> dest [style=dashed]
+//   src [label="..." color=white]
+//   dest [label="..." color=white]
+//   "resources.super" [style=dashed]
+//   allSourceFiles [color=white]
+// }
+// ....
+
+
 /** Usage
 
 > mill run
@@ -42,7 +62,7 @@ Inputs:
 
 // Above, `def lineCount` is a new build target we define, which makes use of
 // `allSourceFiles` (an existing target) and is in-turn used in our override of
-// `resources` (also an existing target). `os.read.lines` and `os.write `come
+// `resources` (also an existing target). `os.read.lines` and `os.write come
 // from the https://github.com/com-lihaoyi/os-lib[OS-Lib] library, which is
 // bundled with Mill. This generated file can then be
 // loaded and used at runtime, as see in the output of `mill run`
@@ -58,9 +78,9 @@ Inputs:
 // your IDE can always help you find the final override of any particular build
 // target as well as where any overriden implementations may be defined.
 //
-// Lastly, custom user-defined targets in Mill benefit from all the same things
-// that built-in targets do: caching, parallelism (with the `-j`/`--jobs`
-// flag), inspectability via `show`/`inspect`, and so on.
+// Unlike normal methods, custom user-defined targets in Mill benefit from all
+// the same things that built-in targets do: automatic caching, parallelism
+// (with the `-j`/`--jobs` flag), inspectability (via `show`/`inspect`), and so on.
 //
 // While these things may not matter for such a simple example that runs
 // quickly, they ensure that custom build logic remains performant and

example/cross/1-simple/build.sc

@@ -9,6 +9,33 @@ trait FooModule extends Cross.Module[String] {
   def sources = T.sources(millSourcePath)
 }
 
+// [graphviz]
+// ....
+// digraph G {
+//   rankdir=LR
+//   node [shape=box width=0 height=0 style=filled fillcolor=white]
+
+//   subgraph cluster_2 {
+//     label="foo[2.12]"
+//     style=dashed
+//     "foo[2.12].suffix" -> "foo[2.12].bigSuffix"
+//     "foo[2.12].sources"
+//   }
+//   subgraph cluster_1 {
+//     label="foo[2.11]"
+//     style=dashed
+//     "foo[2.11].suffix" -> "foo[2.11].bigSuffix"
+//     "foo[2.11].sources"
+//   }
+//   subgraph cluster_0 {
+//     label="foo[2.10]"
+//     style=dashed
+//     "foo[2.10].suffix" -> "foo[2.10].bigSuffix"
+//     "foo[2.10].sources"
+//   }
+// }
+// ....
+
 // Cross modules defined using the `Cross[T]` class allow you to define
 // multiple copies of the same module, differing only in some input key. This
 // is very useful for building the same module against different versions of a

example/cross/3-outside-dependency/build.sc

@@ -11,6 +11,33 @@ def bar = T { s"hello ${foo("2.10").suffix()}" }
 
 def qux = T { s"hello ${foo("2.10").suffix()} world ${foo("2.12").suffix()}" }
 
+// [graphviz]
+// ....
+// digraph G {
+//   rankdir=LR
+//   node [shape=box width=0 height=0 style=filled fillcolor=white]
+//   subgraph cluster_2 {
+//     label="foo[2.12]"
+//     style=dashed
+//     "foo[2.12].suffix"
+//   }
+//   subgraph cluster_1 {
+//     label="foo[2.11]"
+//     style=dashed
+//     "foo[2.11].suffix"
+//   }
+//   subgraph cluster_0 {
+//     label="foo[2.10]"
+//     style=dashed
+//     "foo[2.10].suffix"
+//   }
+//   "foo[2.10].suffix" -> "bar"
+//   "foo[2.10].suffix" -> "qux" [constraint=false]
+//   "foo[2.12].suffix" -> "qux"
+// }
+// ....
+
+
 // Here, `def bar` uses `foo("2.10")` to reference the `"2.10"` instance of
 // `FooModule`. You can refer to whatever versions of the cross-module you want,
 // even using multiple versions of the cross-module in the same target as we do

example/cross/4-cross-dependencies/build.sc

@@ -13,6 +13,47 @@ trait BarModule extends Cross.Module[String] {
   def bigSuffix = T { "[[[" + foo(crossValue).suffix() + "]]]" }
 }
 
+// [graphviz]
+// ....
+// digraph G {
+//   rankdir=LR
+//   node [shape=box width=0 height=0 style=filled fillcolor=white]
+//   subgraph cluster_0 {
+//     label="foo[2.10]"
+//     style=dashed
+//     "foo[2.10].suffix"
+//   }
+//   subgraph cluster_1 {
+//     label="foo[2.11]"
+//     style=dashed
+//     "foo[2.11].suffix"
+//   }
+//   subgraph cluster_2 {
+//     label="foo[2.12]"
+//     style=dashed
+//     "foo[2.12].suffix"
+//   }
+//   subgraph cluster_3 {
+//     label="bar[2.10]"
+//     style=dashed
+//     "bar[2.10].bigSuffix"
+//   }
+//   subgraph cluster_4 {
+//     label="bar[2.11]"
+//     style=dashed
+//     "bar[2.11].bigSuffix"
+//   }
+//   subgraph cluster_5 {
+//     label="bar[2.12]"
+//     style=dashed
+//     "bar[2.12].bigSuffix"
+//   }
+//   "foo[2.10].suffix" -> "bar[2.10].bigSuffix"
+//   "foo[2.11].suffix" -> "bar[2.11].bigSuffix"
+//   "foo[2.12].suffix" -> "bar[2.12].bigSuffix"
+// }
+// ....
+
 // Rather than pssing in a literal `"2.10"` to the `foo` cross module, we pass
 // in the `crossValue` property that is available within every `Cross.Module`.
 // This ensures that each version of `bar` depends on the corresponding version

example/cross/5-multiple-cross-axes/build.sc

@@ -17,6 +17,56 @@ trait FooModule extends Cross.Module2[String, String] {
 
 def bar = T { s"hello ${foo("2.10", "jvm").suffix()}" }
 
+// [graphviz]
+// ....
+//  digraph G {
+//   rankdir=LR
+//   node [shape=box width=0 height=0 style=filled fillcolor=white]
+//
+//   subgraph cluster_6 {
+//     label="foo[2.12,native]"
+//     style=dashed
+//     "foo[2.12,native].suffix"
+//   }
+//
+//   subgraph cluster_3 {
+//     label="foo[2.10,js]"
+//     style=dashed
+//     "foo[2.10,js].suffix"
+//   }
+//   subgraph cluster_4 {
+//     label="foo[2.11,js]"
+//     style=dashed
+//     "foo[2.11,js].suffix" -> "foo[2.10,js].suffix" [style=invis]
+//   }
+//   subgraph cluster_5 {
+//     label="foo[2.12,js]"
+//     style=dashed
+//     "foo[2.12,js].suffix" -> "foo[2.11,js].suffix" [style=invis]
+//   }
+//
+//   subgraph cluster_0 {
+//
+//     label="foo[2.10,jvm]"
+//     style=dashed
+//     "foo[2.10,jvm].suffix"
+//
+//   }
+//   subgraph cluster_1 {
+//     label="foo[2.11,jvmcomm"
+//     style=dashed
+//     "foo[2.11,jvm].suffix" -> "foo[2.10,jvm].suffix" [style=invis]
+//   }
+//   subgraph cluster_2 {
+//     label="foo[2.12,jvm]"
+//     style=dashed
+//     "foo[2.12,jvm].suffix" -> "foo[2.11,jvm].suffix" [style=invis]
+//   }
+//
+//   "foo[2.10,jvm].suffix" -> bar
+// }
+// ....
+//
 // This example shows off using a for-loop to generate a list of
 // cross-key-tuples, as a `Seq[(String, String)]` that we then pass it into the
 // `Cross` constructor. These can be referenced from the command line as shown

example/tasks/1-task-graph/build.sc

@@ -23,6 +23,20 @@ def assembly = T {
   PathRef(T.dest / s"assembly.jar")
 }
 
+// This code defines the following task graph, with the boxes being the tasks
+// and the arrows representing the _data-flow_ between them:
+//
+// [graphviz]
+// ....
+// digraph G {
+//   rankdir=LR
+//   node [shape=box width=0 height=0 style=filled fillcolor=white]
+//   sources -> compile -> assembly
+//   resources -> assembly
+//   mainClass -> assembly
+// }
+// ....
+//
 // This example does not use any of Mill's builtin support for building Java or
 // Scala projects, and instead builds a pipeline "from scratch" using Mill
 // tasks and `javac`/`jar`/`java` subprocesses. We define `T.source` folders,
@@ -50,7 +64,40 @@ My Example Text
 // necessary, depending on what input sources changed:
 //
 // * If the files in `sources` change, it will re-evaluate
-//  `compile`, and `assembly`
+//  `compile`, and `assembly` (red)
+//
+// [graphviz]
+// ....
+// digraph G {
+//   rankdir=LR
+//   node [shape=box width=0 height=0 style=filled fillcolor=white]
+//   sources -> compile -> assembly
+//   resources -> assembly
+//   mainClass -> assembly
+//   assembly [fillcolor=lightpink]
+//   sources [fillcolor=lightpink]
+//   compile [fillcolor=lightpink]
+//   resources [fillcolor=lightgreen]
+//   mainClass [fillcolor=lightgreen]
+// }
+// ....
+//
+// * If the files in `resources` change, it will only re-evaluate `assembly` (red)
+//   and use the cached output of `compile` (green)
 //
-// * If the files in `resources` change, it will only re-evaluate `assembly`
-//   and use the cached output of `compile`
+// [graphviz]
+// ....
+// digraph G {
+//   rankdir=LR
+//   node [shape=box width=0 height=0 style=filled fillcolor=white]
+//   sources -> compile -> assembly
+//   resources -> assembly
+//   mainClass -> assembly
+//   assembly [fillcolor=lightpink]
+//   resources [fillcolor=lightpink]
+//   compile [fillcolor=lightgreen]
+//   sources [fillcolor=lightgreen]
+//   mainClass [fillcolor=lightgreen]
+// }
+// ....
+//