Generators

Rug generators create new projects from an existing source project, where the source project itself is a working project in its own right. A Rug generator has two major components: the “model” project and the modifications needed to transform the model project into a new project. The model project can be any working project you want to use to create new projects. The transformations are encoded in the Rug generator script located under the project’s .atomist directory. Using these components, a generator does the followings:

  1. Copy the content of its host project, .atomist directory (and anything listed in .atomist/ignore) excluded, into the new target directory
  2. Runs the generator’s populate function against the contents of target directory

Let’s look more closely at what makes a project a Rug generator.

Anatomy of a Generator

Suppose we have a model project our team clones to quickly get the skeleton of a Spring Bot Rest Service. The contents of the model project are the following.

~/workspace/spring-boot-rest-basic
    ├── .gitignore
    ├── pom.xml
    ├── README.md
    ├── src
    │   ├── main
    │   │   ├── java
    │   │   │   └── com
    │   │   │       └── company
    │   │   │           ├── HomeController.java
    │   │   │           ├── MyRestServiceApplication.java
    │   │   │           └── MyRestServiceConfiguration.java
    │   │   └── resources
    │   │       ├── application.properties
    │   │       └── logback.xml
    │   └── test
    │       └── java
    │           └── com
    │               └── company
    │                   ├── MyRestServiceApplicationTests.java
    │                   ├── MyRestServiceOutOfContainerIntegrationTests.java
    │                   └── MyRestServiceWebIntegrationTests.java

To convert out model project into a Rug generator, we can use the ConvertExistingProjectToGenerator Rug editor in rug-rugs to add all the necessary directories and files:

$ rug edit atomist:rug-rugs:ConvertExistingProjectToGenerator \
    archiveName=spring-boot-service \
    groupId=company-rugs \
    version=0.13.0 \
    generatorName=NewSpringBootService \
    description="Rug generator for a Spring Boot REST service"
Processing dependencies
  Downloading atomist/rug-rugs/maven-metadata.xml ← rugs (740 bytes) succeeded
  Downloading atomist/rug-rugs/maven-metadata.xml ← global (740 bytes) succeeded
  Downloading atomist/rug-rugs/0.14.0/rug-rugs-0.14.0.pom ← rugs (635 bytes) succeeded
  Downloading atomist/rug-rugs/0.14.0/rug-rugs-0.14.0-metadata.json ← rugs (14 kb) succeeded
  Downloading atomist/rug-rugs/0.14.0/rug-rugs-0.14.0.zip ← rugs (194 kb) succeeded
Resolving dependencies for atomist:rug-rugs:latest completed
Loading atomist:rug-rugs:0.14.0 into runtime completed
  TypeScript files added, run `cd .atomist && npm install`

Running editor ConvertExistingProjectToGenerator of atomist:rug-rugs:0.14.0 completed

→ Project
  ~/workspace/spring-boot-rest-basic (14 kb in 20 files)

→ Changes
  ├── .atomist/package.json created (57 bytes)
  ├── .atomist/tsconfig.json created (627 bytes)
  ├── .atomist/.gitignore created (27 bytes)
  ├── .atomist/generators/NewSpringBootService.ts created (602 bytes)
  ├── .atomist/tests/NewSpringBootService.rt created (153 bytes)
  ├── .atomist/generators/NewSpringBootService.ts updated (580 bytes)
  ├── .atomist/generators/NewSpringBootService.ts updated (583 bytes)
  ├── .atomist/generators/NewSpringBootService.ts updated (584 bytes)
  ├── .atomist/tests/project/NewSpringBootService.rt updated (155 bytes)
  ├── .atomist/tests/project/NewSpringBootService.rt updated (880 bytes)
  └── .atomist.yml created (2 kb)

The groupId and archiveName parameters, coupled with the name of the Rug generator, define the fully-qualified name of the Rug archive (the published package of a Rug).

Once this is completed, the project should look like this:

~/workspace/spring-boot-rest-basic
    ├── .atomist
    │   ├── generators
    │   │   └── NewSpringBootService.ts
    │   ├── .gitignore
    │   ├── package.json
    │   ├── tests
    │   │   ├── project
    │   │   │   ├── NewSpringBootService.feature
    │   │   └── └── Steps.ts
    │   └── tsconfig.json
    ├── .atomist.yml
    ├── .gitignore
    ├── pom.xml
    ├── .project
    ├── README.md
    ├── src
    │   ├── main
    │   │   ├── java
    │   │   │   └── com
    │   │   │       └── company
    │   │   │           ├── HomeController.java
    │   │   │           ├── MyRestServiceApplication.java
    │   │   │           └── MyRestServiceConfiguration.java
    │   │   └── resources
    │   │       ├── application.properties
    │   │       └── logback.xml
    │   └── test
    │       └── java
    │           └── com
    │               └── company
    │                   ├── MyRestServiceApplicationTests.java
    │                   ├── MyRestServiceOutOfContainerIntegrationTests.java
    │                   └── MyRestServiceWebIntegrationTests.java

The .atomist directory contains a metadata file, package.json, defining characteristics of the project and declaring TypeScript dependencies, and has the Rug generator script and its associated test in appropriate subdirectories.

Because all of the Atomist files are hidden under the .atomist directory, our generator project is still a fully functioning, perfectly valid Spring Boot project.

Let’s take a close look at the Rug generator script.

A Basic Generator Script

The generator script’s populate method is invoked after the model project’s files have been copied to the target project. The default contents of the generator script we added above look like the following:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
import { PopulateProject } from '@atomist/rug/operations/ProjectGenerator';
import { Project } from '@atomist/rug/model/Project';
import { Pattern } from '@atomist/rug/operations/RugOperation';
import { Generator, Parameter, Tags } from '@atomist/rug/operations/Decorators';

@Generator("NewSpringBootService", "Rug project for Spring Rest Services")
@Tags("documentation")
export class NewSpringBootService implements PopulateProject {

    populate(project: Project) {
        console.log(`Creating ${project.name}`);
    }
}

export const newSpringBootService = new NewSpringBootService();

After importing the TypeScript Rug typings for the elements we will be using (lines 1–4), we use a decorator to declare the following class a generator (line 6). The first argument to the @Generator decorator is the name of the generator. This is the externally visible and discoverable name of the Rug. This name, along with the generators group and repository, form the fully-qualified name of the generator. The second argument to the @Generator decorator is a brief description of the generator. On line 7 we use the @Tags decorator to apply some tags to our generator so people can search for it more easily. Using the @Tags decorator is optional but highly recommended.

We define the class that will implement our generator on line 8. A generator implements the PopulateProject interface. This interface requires the populate(Project) method to be defined, which we do on line 10 (more on that below). It is convention for the generator and the class that implements it to have the same name.

In the last line of the generator script we export an instance of that generator to make it visible to the Rug runtime when it executed (line 15). Like the generator class name, the name of the const does not matter, but it is convention to use the generator/class name, lower-casing the first letter.

As explained earlier, a generator copies the content of the project where it lives into a target directory before applying changes. The definition of our generator currently performs only the copy (this is done automatically for us). Let’s now amend the generator to modify the copied contents, for example to change the name of the copied class. An action users would likely do manually.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
import { PopulateProject } from '@atomist/rug/operations/ProjectGenerator';
import { Project } from '@atomist/rug/model/Project';
import { File } from '@atomist/rug/model/File';
import { Pattern } from '@atomist/rug/operations/RugOperation';
import { Generator, Parameter, Tags } from '@atomist/rug/operations/Decorators';
import { PathExpressionEngine } from '@atomist/rug/tree/PathExpression';

@Generator("NewSpringBootService", "Rug project for Spring Rest Services")
@Tags("documentation")
export class NewSpringBootService implements PopulateProject {

    @Parameter({
        displayName: "Class Name",
        description: "name for the service class",
        pattern: Pattern.java_class,
        validInput: "a valid Java class name, which contains only alphanumeric characters, $ and _ and does not start with a number",
        minLength: 1,
        maxLength: 50,
        required: false
    })
    service_class_name: string;

    populate(project: Project) {
        console.log(`Creating ${project.name}`);

        let eng: PathExpressionEngine = project.context.pathExpressionEngine
        eng.with<File>(project, '/src//File()[contains(@name, "MyRestService")]', f => {
            f.replace("MyRestService", this.service_class_name);
            f.setPath(f.path.replace("MyRestService", this.service_class_name));
        });
    }
}

export const newSpringBootService = new NewSpringBootService();

Rugs, like typical methods, often take parameters to customize their behavior. Generators have a required parameter: the name of the project that will be generated. The project name parameter is automatically defined for a generate. All other parameters used by the generator must be declared. Parameters are declared using the @Parameter decorator (line 10). The @Parameter decorator provides the metadata for the parameter while the subsequent instance variable declaration provides the name and default value, if any. The @Parameter decorator accepts a single argument, a JavaScript object. The JavaScript object passed to @Parameter accepts all of the property names shown above, but only pattern is mandatory. The pattern property provides an anchored regular expression used to validate user input. Here we use one the Atomist pre-defined Patterns (line 13). Despite the fact that the other @Parameter properties are option, it is highly recommended to provide them to help consumers of your generator.

The populate method takes a single argument, a Project object. The Project provided to this method contains the contents of the generated project, i.e., all the files copied from the generator project. Using this object, you can alter the exact copy of the original project as appropriate so the result is the new project with the desired contents. To effect your desired changes, you have the power of TypeScript and the Rug programming model.

In that regards, as Atomist comprehends filesystem and code structure, the Rug programming model offers a powerful mechanism to make the above example a lot less brittle through path expressions. In this generator script, we query the filesystem for all files containing a specific token in their names (line 27). Then for each one of these files, we replace its content (line 28) and move it to different path (line 29).

Rugs should be tested as any other pieces of software, Rug and its runtime natively supports a BDD-centric testing approach, based on the Gherkin DSL.

The test for our generator could be described as follows in .atomist/tests/project/NewSpringBootService.feature:

Feature: Creating new Spring Rest Service projects

Scenario: A default Spring Rest project structure should be generated
 Given an empty project

 When running the Spring Boot Service generator

 Then the name of the application file is changed
 Then the name of the configuration file is changed
 Then the name of the application tests file is changed
 Then the name of the integration tests file is changed
 Then the name of the web integration tests file is changed

 Then the name of the class in the application file is changed
 Then the name of the class in the configuration file is changed
 Then the name of the class in the application tests file is changed
 Then the name of the class in the integration tests file is changed
 Then the name of the class in the web integration tests is changed

Implemented by the steps in .atomist/tests/project/Steps.ts file:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
import { Given, When, Then, ProjectScenarioWorld } from "@atomist/rug/test/project/Core";
import { Result } from "@atomist/rug/test/Result";
import { Project } from "@atomist/rug/model/Project";

Given("an empty project", p => {})

When("running the Spring Boot Service generator", (p: Project, world: ProjectScenarioWorld) => {
  let generator = world.generator("NewSpringBootService");
  world.generateWith(generator, {"service_class_name": "CalendarService"});
})

Then("the name of the application file is changed", (p: Project) =>
    p.fileExists("src/main/java/com/company/CalendarServiceApplication.java")
)
Then("the name of the configuration file is changed", (p: Project) =>
    p.fileExists("src/main/java/com/company/CalendarServiceConfiguration.java")
)
Then("the name of the application tests file is changed", (p: Project) =>
    p.fileExists("src/test/java/com/company/CalendarServiceApplicationTests.java")
)
Then("the name of the integration tests file is changed", (p: Project) =>
    p.fileExists("src/test/java/com/company/CalendarServiceOutOfContainerIntegrationTests.java")
)
Then("the name of the web integration tests file is changed", (p: Project) =>
    p.fileExists("src/test/java/com/company/CalendarServiceWebIntegrationTests.java")
)

Then("the name of the class in the application file is changed", (p: Project) =>
    p.findFile("src/main/java/com/company/CalendarServiceApplication.java").contains("CalendarServiceApplication")
)
Then("the name of the class in the configuration file is changed", (p: Project) =>
    p.findFile("src/main/java/com/company/CalendarServiceConfiguration.java").contains("CalendarServiceConfiguration")
)
Then("the name of the class in the application tests file is changed", (p: Project) =>
    p.findFile("src/test/java/com/company/CalendarServiceApplicationTests.java").contains("CalendarServiceApplicationTests")
)
Then("the name of the class in the integration tests file is changed", (p: Project) =>
    p.findFile("src/test/java/com/company/CalendarServiceOutOfContainerIntegrationTests.java").contains("CalendarServiceOutOfContainerIntegrationTests")
)
Then("the name of the class in the web integration tests is changed", (p: Project) =>
    p.findFile("src/test/java/com/company/CalendarServiceWebIntegrationTests.java").contains("CalendarServiceWebIntegrationTests")
)

If you’re not familiar with this approach, the .atomist/tests/project/NewSpringBootService.feature describes our tests in a set of hypotheses and expectations. All those steps are implemented in the .atomist/tests/project/Steps.ts file which is executed when the test is run:

$ rug test
Resolving dependencies for com.company.rugs:spring-boot-service:0.13.0:local completed
Invoking TypeScript Compiler on ts script sources completed
Loading com.company.rugs:spring-boot-service:0.13.0:local completed
  Executing feature Creating new Spring Rest Service projects
    Executing test scenario A default Spring Rest project structure should be generated
  Creating project_name
Running tests in com.company.rugs:spring-boot-service:0.13.0:local completed

Successfully executed 1 of 1 test: Test SUCCESS

Assuming we change one of the hypotheses to make it fail, rug would notify us with a relevant error message:

$ rug test
Resolving dependencies for com.company.rugs:spring-boot-service:0.13.0:local completed
Invoking TypeScript Compiler on ts script sources completed
Loading com.company.rugs:spring-boot-service:0.13.0:local completed
  Executing feature Creating new Spring Rest Service projects
    Executing test scenario A default Spring Rest project structure should be generated
  Creating project_name
Running tests in com.company.rugs:spring-boot-service:0.13.0:local completed

→ Test Report
  Failures
  └─┬ Creating new Spring Rest Service projects
    └─┬ A default Spring Rest project structure should be generated
      ├─┬ the name of the application file is changed: Failed
      | └── function (p) {
    return p.fileExists("src/main/java/com/company/CalndarServiceApplication.java");
}
      ├── the name of the application tests file is changed: Passed
      ├── the name of the class in the application file is changed: Passed
      ├── the name of the class in the application tests file is changed: Passed
      ├── the name of the class in the configuration file is changed: Passed
      ├── the name of the class in the integration tests file is changed: Passed
      ├── the name of the class in the web integration tests is changed: Passed
      ├── the name of the configuration file is changed: Passed
      ├── the name of the integration tests file is changed: Passed
      └── the name of the web integration tests file is changed: Passed

Unsuccessfully executed 1 of 1 test: Test FAILURE

As you can see, Rug generator scripts are simple functions that apply changes against a freshly copy of your its content. This changes may be parametarized to tailor the result to the user’s expectations. Finally, following a test-driven approach, generators can be quickly validated before being released.