Simplify unit testing w/ different property values

[eeverman]

Is your feature request related to a problem? Please describe.
Setting up JUnit tests is still somewhat verbose:

@KillAndHowBeforeThisTest
@Test
public void myTest() {
  AndHow.findConfig().setClasspathFilePath("/some/path/to/a/file.properties);
}

Describe the solution you'd like
Create some dedicated test annotations that accept properties files as arguments:

@ReloadAndHowFromClasspathProperties(path="/some/path/to/a/file.properties");
@Test
public void myTest() {
  // Test uses AndHow loaded from the prop file (but still uses any discovered AndHow init classes)
}

@ReloadAndHowFromClasspathPropertiesOnly(path="/some/path/to/a/file2.properties");
@Test
public void myTest2() {
  // Test uses AndHow loaded from the prop file (skipping any AndHow init classes)
}

These annotations could be applied to either test classes or individual test methods. The 'Only' method could also shut down the other loaders, perhaps other than the fixedValue loader to allow tests to override individual values.

Describe alternatives you've considered
Method naming could use some consideration.

[eeverman]

Maybe these method signatures look like this:

@ConfigFromFile(path="test1.properties") //Relative path next to this test class
@ConfigFromFile(path="/org/project/package/test.properties) //Abs class path
@ConfigFromFileSameNameAsClass //Look for a file w/ the same name as the [test class].properties
@ConfigFromFileSameNameAsMethod//Look for a file w/ the same name as the [test class]-[test method].properties

An additional possible feature: Overlay a config file:

@ConfigOverlayFrom...(path) //Override only some properties by overlaying an additional properties file

This could be done by inserting a loader before the Std PropFile loader.

During unit testing, auto-discovery of an AndHowConfig instance is not useful if its possible to specify the entire property state w/ a specific properties file - it would just make it harder to decipher what the configured state is during the test. Thus, I think these annotations should all turn off AndHowConfig discovery. If you are not using these annotations, then it may be useful to auto-discover AndHowConfig to provide a universal config state for testing.

Along with turning off AndHowConfig Auto-discovery, this should probably also disable other loaders that make testing sensitive to the environment. Each of the loaders listed below have the potential to override a value that might be configured in the specified properties file and thus should be turned off:

• StdSysPropLoader
• StdEnvVarLoade
• StdJndiLoader

The FixedValue loader should be kept, since it would be handy to override a specific value for a specific test.
The StringArgs loader should be kept because a test might call a classes main arguments directly and expect its arguments to be respected.
The StdPropFileOnFilesystemLoader loader should be kept because it would only be used explicitly, i.e., the user would know they are doing it.

[eeverman]

A few more considerations:

• Currently the Config annotations do not initialize AndHow. This seems fine for the BeforeEach and BeforeThisTest versions, but is problematic for the BeforeAll b/c a test may modify the configuration, creating a test order dependency that sometimes works and sometimes causes an error (if attempting to configure after init). It would be better for the BeforeAll to call AndHow.instance() before the first test, but is it possible to do this and still allow the user to augment the config in a BeforeAll method?
  • There is a BeforeTestExecutionCallback that the ConfigBeforeAll could use. Usage might look like this:
  • ConfigBeforeAll places a marker in the test class storage during BeforeAll, marking the state as needing AndHow initialized
  • Other test method annotations place a marker in test storage during BeforeEach, marking the test as either OK (default) or not OK to initialize AndHow beforehand, using a shared interface, e.g. TestMethodInitAllowed.
  • During ConfigBeforeAll.BeforeTestExecutionCallback a check is done of the class marker: Does this class still need AndHow initialized? If yes, check the TestMethodInitAllowed marker to see if this method is appropriate for AndHow initialization. If it is, initialize AndHow and remove the marker indicating that the AndHow still requires initialization.
  • Other annotations, such as KillAndHow will need to participate as well.
• How do the Config annotation interact w/ the KillAndHow ann's?
• Need to allow spec'ing classesInScope as Strings to handle private classes
• Nice to also add classes to exclude (Tricky - no support for this in AndHow, so may need to be separate feature)
• From JUnit 'Gitter' convo: Try using: AnnotationSupport.findAnnotation(context.getRequiredTestClass(), A) to find the annotation instead of reflect methods. Looking at what this method does, it may fix an issue that happens when the annotation is on a super class: Normal methods won't find that, but the util class will hunt through the JUnit inheritance model to find the annotations that apply.
• Would make sense to have a base extension class w/ an abstract method that returns an enum of the type of annotation, which could be used to auto-determine which storage the extension needs. If an extension left a marker as to its type in the appropriate storage, other extensions could find conflicts, for instance, CONFIG_THIS_TEST and KILL_THIS_TEST are not compatible:
  • CONFIG_EACH_TEST
  • CONFIG_ALL_TESTS
  • CONFIG_THIS_TEST
  • KILL_EACH_TEST
  • KILL_ALL_TESTS
  • KILL_THIS_TEST

Testing Notes

• Need to test inheritance of test classes
• The inheritance of annotations (putting the config annotation on a new annotation) should be tested
• Also test adding classes that don't have Properties to the list of inScope classes, and classes that have inner-classes with no Properties. Shouldn't cause an error.
• What happens if 'null' is assigned to the includedClasses array?
• Need to test earlier versions of JUnit
• Need to test relative paths of config file from a parent class in a different package. The relative path should be relative to the superclass, not the subclass.