TestFramework

From ADempiere
Revision as of 18:59, 8 February 2011 by Red1 (Talk) (Tips: ensure javac debug = true / on)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search
This Wiki is read-only for reference purposes to avoid broken links.
Note.gif Note:

There is a good upgrade to Slim by Fitnesse and it may deprecate the approach here depending on what you really want to do. Please see FitnesseSlim.

Carlos Ruiz and Joel Stangeland have been working together towards a framework for building repeatable data driven test cases for base functionality. We researched it and Carlos took it to the point of a POC. Below are the findings of the POC. We estimate there are about 40 hours or so to develop the complete functionality and would be happy to work to finish it if there is community interest.

Fitnesse Integration

Work just started - we'll use this page to notify about the advance and put technical info, samples, howtos, etc.

You can push the watch button on top of this page to be notified about changes via e-mail if this is set up in your Preferences

Tasks done

Pending Tasks

  • Explore the best way to implement test suites, cases, etc
  • Document design
  • Develop solution
  • Implement repository for test cases
  • Implement svn integration with repository for test cases

Some future TODOs

  • Explore about remote fitnesse (this feature allows you to use the testadempiere test cases in your own fitnesse installation)
  • Implement thousands of test cases

Done Fixtures

  • fixture to log in
  • fixture to create record - at the end every column is set on a context variable
  • fixture to run a process or workflow - it can read context variables
  • fixture to set doc action of a document (calling completeIt, prepareIt, voidIt, etc) - it can read context variables

Ideas for new Fixtures

  • fixture to create/update record
  • fixture to update record
  • fixture to delete record
  • fixture to check pass/fail (assertions)
  • fixture to execute arbitrary sql
  • fixture to set up a variable executing arbitrary SQL
  • fixture to set context variables
  • let some fixtures generate random values for some fields (i.e. Value)

Repository

You can check, download, test the code from here:

http://adempiere.svn.sourceforge.net/viewvc/adempiere/branches/fitnesseIntegration/

Playground

A playground environment was set up here:

http://www.testadempiere.com:8082/ (doesn't work now - Redhuan D. Oon)

Some sample test cases are being worked here:

http://www.testadempiere.com:8082/AdempiereTest (doesn't work now - Redhuan D. Oon 02:48, 11 January 2011 (UTC))

Fitnesse Tips

  • In the main page you must define the variable COMMAND_PATTERN to let fitnesse know the parameters passed to the JVM, for example: !define COMMAND_PATTERN {java -Xms32m -Xmx512m -DPropertyFile=/home/admin/fitnesse/AdempiereTest.properties -DADEMPIERE_HOME=/home/admin/fitnesse -DLOG4J_LEVEL=WARNING -cp %p %m}
  • You can add -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=1044 to the COMMAND_PATTERN to allow remote debugging on port 1044 (the port can be changed to any port you prefer)
  • The fixtures interpret the variable LOG4J_LEVEL to set the log level of Adempiere

The Concept

the original idea: http://sourceforge.net/forum/message.php?msg_id=4051192 Fitnesse (based/forked from fit) has the mechanisms to describe input data, test case and output expected data - and as is written in java I think is easy to integrate with Adempiere. And in a really easy and collaborative way - just adding test cases in a wiki site.

The POC

The POC is just an initial review to see if the tool can support the need expressed. It is based on the open source user testing tool called FitNesse. These are the findings of the POC.

  1. the tool looks really powerful (I'll show you below) - and with little effort we can integrate it with Adempiere
  2. with proper "fixtures" the tool could really allow non-technical people to define test cases within Adempiere

The tool has very good things already thought for test cases, most prominent things I found are:

  1. Wiki based - users define the data - test cases - and expected output in wiki tables - the format of these wiki tables is easier than mediawiki, and it has a "formatter" for tables copied from spreadsheet. This is something good - user can combine text with tables - tables are the description of the data, the rest of the wiki page can be just prose to describe the test case for readers.
  2. it's intended for users/customers to define AcceptanceTests -> http://fitnesse.org/FitNesse.AcceptanceTests
  3. it has some pre-supported format for test tables -> http://fitnesse.org/FitNesse.TestTableStyles and the test tables can be extended to achieve specific goals - in the POC I extended TableFixture to make an AdempiereTableFixture
  4. the concept of a test case is easy - suggested approach is called to define the BuildOperateCheck (in XP ArrangeActAssert), this is you define how to Build the test data, then define how to Operate the test data, and finally Check the results -> http://fitnesse.org/FitNesse.BuildOperateCheck All the three are defined in a wiki page and conform a test case, but there is more! You can define a common Build for several test cases (via include keyword), and even more! you can define a common Build that uses variable parameters from the page.
  5. Also found something really good -> http://fitnesse.org/FitNesse.TestDevelopmentEnvironment This is, you can have test cases defined in a centralized wiki server, and the distributed developers can have a local version and point to the server with a virtual wiki, and run the tests locally!
  6. well, last but not least - the tool is open source  ;-) - java based, and the code is not so big - and it looks to have a good community around

OK, let's stop talking about the tool and let's review the explained screenshots of the POC.

Summarizing - I wrote a generic an AdempiereFixture to create data in Adempiere. I used GenericPO to allow generic definition of the table and columns to be filled. You can see a more detailed description with screenshots in the attached document. And attached you can see also the code - very little code indeed.

_____________________

POSSIBILITIES FOR A COMPLETE IMPLEMENTATION

What I'm thinking initially is that we can improve the fixture (or implement several fixtures) to cope with more "business cases" of Adempiere.

Fixture to create/update records: - The fixture shown allows the creation of any record in the database using the GenericPO - it can be enhanced to use the proper model class associated to the table (i.e. MBPartner in this case) - The fixture currently just create the record - I'm thinking how to write a fixture to allow update if exists. As you see I defined some "keywords" for the fixture -> ** table ** to describe the name of the table to be inserted - and ** save ** to let the fixture call the save event on the PO.

This can be extended to something more useful, i.e. a Fixture that defines in the first column an action and on the next column(s) the data for the action, something like:

|setTable  | C_BPartner |
|setName | Carlos |
|set...
| set_Value | Comments | Comments here |

the last one could be calling a generic setter to avoid validations, or to fill columns not in the X_ class.

Another important thing I think is to allow definition of the keys indirectly, instead of:

| C_BP_Group_ID | 103 |

we must define something like

| C_BP_Group_ID->C_BP_Group.Name | Standard |

Fixture to instance a document: We can define on the same fixture - or in a different one the ability to instance a document, this is to invoke the normal methods of a document (completeIt, voidIt, prepareIt, etc) Something like this could be implemented - but it needs further thinking:

| setDocument | C_Invoice | 1000000 |
| completeIt |

Fixture to run a process: Something like this could be defined in the same or a new fixture:

| setProcess | CopyInvoice |
| setRecord_ID | 1000000 |
| setParameter | DocStatus | CO |

Fixture to examine data: We can construct a new fixture (or the same) to examine data - but possibly the first one can serve for the same purpose. Or to execute a SQL command against the database to look for the expected result.

Well - adding SQL is going far from the original purpose that allows any user to define test cases - but that could be an add-on - and we could try to find other ways for the user to find the proper records.


That's the main idea I'm having with this - I think it's not so hard - a better working thing can be constructed maybe in 40h.

Tips

Temporary section by Redhuan D. Oon. Will merge to Carlos notes above.

Debugging

  • Remote Debugging can be done by creating in your Eclipse, a RemoteDebug configuration listening at port 1044 and putting this in your fitnesse page:
!define COMMAND_PATTERN {java -Xdebug  
   -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=1044 -cp %p %m}
!path /Users/red1/Documents/projects/Fitnesse/fitnesse.jar
  • When doing RemoteDebug ensure your project's build.xml that deploys the jar has a JAVAC DEBUG = "TRUE" or "ON".

Links

If you want to read more information here are the links:

Schuchert Fitnesse - a very good tutorial that incorporates latest Slim improvement against Fix.

http://www.fitnesse.org/

http://fitnesse.info/ (I downloaded from here the code and binary)

http://tech.groups.yahoo.com/group/fitnesse/ (support group)

http://www.bandxi.com/fitnesse/index.html (eclipse plugin!)

http://sourceforge.net/forum/forum.php?thread_id=1628494&forum_id=610546 (original thread)

http://sourceforge.net/forum/forum.php?thread_id=2971176&forum_id=610548 (call for sponsorship)

http://sourceforge.net/forum/forum.php?thread_id=3091893&forum_id=610548 (playing with the tool + news)

Example of actual use - Cost Engine/Testing

IRC Meeting Full 20090310

Fitnesse

Fitnesse use in adempiere

Files