IRC Meeting Full 20090310

From ADempiere
Jump to: navigation, search
This Wiki is read-only for reference purposes to avoid broken links.

Invitation: Proposing weekly meeting/2009-03-10 21UTC

AGENDA:

  • TEST FRAMEWORK
    • Explain about fitnesse integration approach
    • Receive feedback about testing framework
    • Suggestions to create test cases, etc

<CarlosRuiz>: Hi
<JoelS>: Hola
<karsten-thiemann>: hope it's the right time now..
<Kai_>: Hi
<CarlosRuiz>: just read there was confusion with the timezone?
<CarlosRuiz>: I saw this weekend the hour moved again in USA - and my USA hour of the post went obsolete
<karsten-thiemann>: :)
<karsten-thiemann>: it's a bit complicated to coordinate such a world wide meeting..
<MCalderon>: Hi
<Kai_>: Next time we should post a link to an UTC clock. :-)
<CarlosRuiz>: :-D
<CarlosRuiz>: OK - we have just 1h
<CarlosRuiz>: the agenda is:
<CarlosRuiz>: TEST FRAMEWORK
<CarlosRuiz>: - Explain about fitnesse integration approach
<CarlosRuiz>: - Receive feedback about testing framework
<CarlosRuiz>: - Suggestions to create test cases, etc
<MCalderon>: I got confused with the time, so I will just let the protocol run. Bye!
<CarlosRuiz>: ok - the intention is to comment you about the work I'm starting and hopefully get your ideas and opinions about how to evolve this
<CarlosRuiz>: right?
<JoelS>: sounds good
<CarlosRuiz>: I installed yesterday fitnesse in the testadempiere site
<karsten-thiemann>: fitnesse seems to be a good collaboration testing tool
<karsten-thiemann>: so I think we should try it with adempiere
<CarlosRuiz>: still very draft - and not the main dev work
<CarlosRuiz>: but I installed it to show you this meeting
<CarlosRuiz>: you can check the site here:
<CarlosRuiz>: http://www.testadempiere.com:8082/
<CarlosRuiz>: that's a typical fitnesse started just out of the box without any configuration
<CarlosRuiz>: I haven't advanced too much in the work - just trying to explore the possibilities
<CarlosRuiz>: until now I created two "Fixtures"
<CarlosRuiz>: fixture in fitnesse is a program that I can construct to drive the wiki test tables
<CarlosRuiz>: there are good samples in the fitnesse documentation if you want to explore
<CarlosRuiz>: the fixtures I created until now are two
<CarlosRuiz>: one to create data
<CarlosRuiz>: and the other to run a process
<CarlosRuiz>: but the idea is to create fixtures to many other things
<CarlosRuiz>: well - maybe not "many"
<CarlosRuiz>: with run a process you can trigger documents workflow
<CarlosRuiz>: fitnesse allow users to create test cases like wiki tables
<CarlosRuiz>: for example take a look to this
<CarlosRuiz>: http://www.testadempiere.com:8082/AdempiereTest.TestCreateBusinessPartner
<CarlosRuiz>: That's the fixture I created to allow creation of data in any table
<CarlosRuiz>: everybody following?  :-) you can interrupt me at any time
<CarlosRuiz>: if you want to check the wiki "code" of the fixture you can push the edit button at the left
<CarlosRuiz>: and you'll see a !define line - with the command to run adempiere
<CarlosRuiz>: that's not needed per page - it could be defined globally
<CarlosRuiz>: the second line
<CarlosRuiz>: !|org.adempiere.fitnesse.AdempiereFixture|
<CarlosRuiz>: mention the fixture code
<CarlosRuiz>: that's the code I wrote - a java class called AdempiereFixture to process the table data
<CarlosRuiz>: the rest of the lines is the table data
<CarlosRuiz>: I defined that first line must describe the "** table **" to insert
<CarlosRuiz>: the rest of the lines will be a pair of "Column | Data"
<CarlosRuiz>: and the last line of the fixture is "** save **" to make the fixture save the data
<CarlosRuiz>: now - after the fixture is written
<CarlosRuiz>: you can push the "Save" button to save it (not needed if you're just reading - just navigate back in the navigator history)
<JoelS>: Carlos- it writes directly to the table, or through the PO?
<CarlosRuiz>: through PO
<CarlosRuiz>: so, it invokes beforeSave / afterSave / ModelValidator methods
<CarlosRuiz>: in a wiki page
<JoelS>: so some tests might have bad data that would fail proper authentication
<CarlosRuiz>: exactly
<JoelS>: others would be expected to pass
<CarlosRuiz>: yes
<Kai_>: And the how much code is in the fixture?
<CarlosRuiz>: a wiki page typically have several tables
<CarlosRuiz>: sorry - I haven't committed still the code - will do soono
<CarlosRuiz>: soon
<CarlosRuiz>: AdempiereFixture is 49 lines
<CarlosRuiz>: there is another fixture to run processes
<CarlosRuiz>: AdempiereProcessFixture -> 328 lines
<CarlosRuiz>: I tested for example with this:
<CarlosRuiz>: http://www.testadempiere.com:8082/AdempiereTest.TestInitialClientSetup
<CarlosRuiz>: a big process to run
<CarlosRuiz>: it creates a new tenant
<CarlosRuiz>: if you want to try I would recomment to copy the code and create a new test page with slightly changed data - to avoid collisions with other testers
<CarlosRuiz>: but - as I said - this is just a draft - the final result could be different
<CarlosRuiz>: when I did the web services I wrote like three versions  :-)
<CarlosRuiz>: before I got satisfied
<CarlosRuiz>: so
<JoelS>: so your fitnesse page just has to have a table with the parameters expected by the process
<CarlosRuiz>: a fitnesse page can have several tables
<CarlosRuiz>: in fact a fitnesse page can be lots of tables tests
<CarlosRuiz>: for example
<CarlosRuiz>: in the same fitnesse page
<CarlosRuiz>: we could create a tenant (calling Initial Client Setup process)
<CarlosRuiz>: then create a product
<CarlosRuiz>: the create a business partner
<CarlosRuiz>: then create a sales order
<CarlosRuiz>: and so on
<CarlosRuiz>: with very few fixtures we can have many things
<Kai_>: And how is the result checked?
<karsten-thiemann>: is there an easy way to drop all created data?
<JoelS>: probably we're not ready to distract into best practices, but with automated tests, is better to have a discreet process, that returns a pass fail
<CarlosRuiz>: I think we just need a few fixtures:
<CarlosRuiz>: - create data
<CarlosRuiz>: - create or update data
<CarlosRuiz>: - run a process
<CarlosRuiz>: - assertions !!!! <----
<JoelS>: so at end of the 'create tenant', if the process completed, it would come back Yes
<CarlosRuiz>: one by one  :-)
<CarlosRuiz>: > And how is the result checked?
<CarlosRuiz>: My idea is to create assertion fixtures
<CarlosRuiz>: to check data
<JoelS>: ah, yes, forgot that
<CarlosRuiz>: or to check data compared with values before the test
<CarlosRuiz>: > is there an easy way to drop all created data?
<CarlosRuiz>: I haven't tested still - but there are some processes called Setup and Tear
<CarlosRuiz>: Setup to run BEFORE the test suit
<CarlosRuiz>: and Tear to run AFTER the test suite (maybe to clean data)
<CarlosRuiz>: I guess Setup could call RUN_ImportAdempiere
<CarlosRuiz>: ah - I mentioned test "suites"
<CarlosRuiz>: I can group several tables in a page
<CarlosRuiz>: and I can group several pages in a suite
<CarlosRuiz>: and run test suites
<CarlosRuiz>: now with Joel - sorry - I didn't understand your comment -> "is better to have a discreet process, that returns a pass fail"
<CarlosRuiz>: I think we can have really few fixtures as I mentioned
<CarlosRuiz>: and focus more in making the fixtures easily usable by "testers"
<CarlosRuiz>: for example: instead of:
<CarlosRuiz>: C_Currency_ID | 230
<CarlosRuiz>: a final user must write something like:
<CarlosRuiz>: Currency | USD
<CarlosRuiz>: ?
<JoelS>: !
<CarlosRuiz>: Joel
<JoelS>: with the automated testing, if you can make a test that returns Pass/Fail, then at the end, you can look to see that all your tests passed
<JoelS>: so even if it is insert data to be, at the end, you do a check to see, did the data insert, and if so, the test result is Pass
<JoelS>: the next step is, that can be checked by the build process, so if a commit breaks a test, a notice can be sent
<JoelS>: done
<CarlosRuiz>: yes - I still haven't tested how - but there is another fitnesse feature to pass data between tables,
<CarlosRuiz>: so - I can create the C_BPartner, get the C_BPartner_ID and on the next table use the variable C_BPartner_ID of the previous table
<CarlosRuiz>: and also I read somewhere fitnesse can be integrated with hudson
<CarlosRuiz>: and other tools
<Kai_>: But could I define somewho a desired result?
<CarlosRuiz>: ok - let's suppose a test case to create a "POS Cash Sales Order"
<CarlosRuiz>: that will be my first goal  :-)
<CarlosRuiz>: I'm imagining something like
<CarlosRuiz>: 1st fixture table - create the sales header
<karsten-thiemann>: kai - look here: http://www.testadempiere.com:8082/FitNesse.TwoMinuteExample
<CarlosRuiz>: 2nd fixture table - create the sales line
<CarlosRuiz>: if additional lines then more fixture tables for the sales order lines
<CarlosRuiz>: 3rd fixture table - invoke "complete order" process
<CarlosRuiz>: and after that
<CarlosRuiz>: one or more "assertion" fixture tables
<CarlosRuiz>: to check
<CarlosRuiz>: 1 - creation of shipment
<CarlosRuiz>: 2 - creation of invoice
<CarlosRuiz>: 3 - creation of cash journal line
<CarlosRuiz>: 4 - the balance of BP must be updated
<CarlosRuiz>: 5 - the qty on hand on warehouse must be diminished by the qty of the sales order line
<CarlosRuiz>: etc
<CarlosRuiz>: is the "assertion" name ok?
<JoelS>: excellent
<karsten-thiemann>: one problem I see are the duplicated key errors
<Kai_>: Ah OK, sorry I was a little bit slow. Yes, "assertion" is great
<CarlosRuiz>: ok - that's my idea - still really draft and hoping to hear feedback
<CarlosRuiz>: not necessarily here - I suppose we're going to think this further and will have some new ideas
<Kai_>: Has someone experince with other test suites?
<Michael_Judd>: the concept is good and presumably - you have had a look at a few tools ....
<karsten-thiemann>: if we like to test the creation and use of a product it's not easy to remove the product from the db after the test
<karsten-thiemann>: so for a second run you might have to drop and recreate the whole db
<Kai_>: Yes Karsten, but where is the problem?
<CarlosRuiz>: we could have a fixture to create and another to create/update
<karsten-thiemann>: it's slow
<CarlosRuiz>: I think is good to have a "just create fixture" precisely to check that adempiere is checking properly unique names or unique keys
<Michael_Judd>: yes - and a lot of work - but what is the alternative ?
<Kai_>: But when I understand it right, finally these are tests which might run over hours.
<CarlosRuiz>: and another fixture to create/update - can allow to reuse data for more testings
<karsten-thiemann>: and maybe a fixture that adds random strings to unique names for mass creation of products etc
<CarlosRuiz>: good point - random data in value / name
<Kai_>: Good question Michael. Are there alternatives?
<CarlosRuiz>: as we write the fixture -we can make it interpret the data in the fixture table as we want
<CarlosRuiz>: I could write code to interpret @RandomNumber@ - @RandomString@ etc
<CarlosRuiz>: ok - I'm lost with the question
<CarlosRuiz>: "but what is the alternative ?"
<Kai_>: I like here the concept, that the test could be wirtten by non-coders.
<Kai_>: Theoretically. :-)
<Michael_Judd>: yes Kai_ - this is a good concept
<karsten-thiemann>: yes - but we need test testers..
<Michael_Judd>: yes - but that is where expertise comes in .... people will review tests like they review code
<Michael_Judd>: many eyeballs etc ...
<Michael_Judd>: it will help us build up more demo data too - to use in the tests perhaps ...
<CarlosRuiz>: sure - I can imagine even a test suite that creates completely a test tenant
<Michael_Judd>: but Carlos is talking about one specific tool - so my question was - do you carlos think this is the best tool ?
<CarlosRuiz>: no Michael - I can't compare as I haven't reviewed many other tools - and I don't have a lot of experience with test frameworks
<Michael_Judd>: so do you propose we use this tool? or do you propose we review available tools ?
<CarlosRuiz>: no, we already chose to use this tool
<CarlosRuiz>: well - I already chose  :-)
<CarlosRuiz>: the history behind is
<JoelS>: i spent a long time researching before picking fitnesse
<Michael_Judd>: ok - so then we have to learn this tool!
<Michael_Judd>: It's ok - I just was asking if someone has laready done the legwork - thanks Joel....
<CarlosRuiz>: yep - that's the history behind this - some time ago Joel wrote a forum post about fitnesse
<Michael_Judd>: so we need to build up tests for different functional areas ...
<CarlosRuiz>: lot of time ago  :-)
<Kai_>: And there are so many: http://java-source.net/open-source/testing-tools
<Kai_>: Good that we already chose. :-)
<CarlosRuiz>: hehehe
<CarlosRuiz>: the tool really is called FIT
<CarlosRuiz>: fitnesse is Fit + wiki + website
<CarlosRuiz>: ok
<CarlosRuiz>: the idea is really simple until now:
<CarlosRuiz>: - fixture to create data
<CarlosRuiz>: - fixture to run a process or workflow
<CarlosRuiz>: - fixture to check pass/fail (assertions)
<CarlosRuiz>: and I think we must follow "KISS" and evolve
<CarlosRuiz>: so - my focus is more about making it usable for non-technical users
<CarlosRuiz>: so - I'm thinking that the create data must allow definition via lookups - not the ID's
<CarlosRuiz>: or for example the Karsten's idea is great - allow random values
<Kai_>: Maybe we find some other useful "system variables".
<CarlosRuiz>: sure - that's the idea of this meeting - to trigger your minds to think and suggest
<CarlosRuiz>: new fixtures - or new uses for the fixtures - or new useful "variables" for the fixtures ... etc
<CarlosRuiz>: and going further
<CarlosRuiz>: how to set up test cases - but I think that will be after I can present something more usable for you
<CarlosRuiz>: so - if you have more questions now - please
<CarlosRuiz>: and if not - please write further questions or suggestions on forum or wiki page
<CarlosRuiz>: the wiki page that I'm going to update to keep community informed is http://adempiere.com/wiki/index.php/TestFramework
<ar_howard>: Sorry I missed this .. bit confused :-) time in Australia, Brisbane is 8:00am 11/3/09 (Australian Eastern Standard Time)
<JoelS>: Carlos, is there a place to start making tests?
<CarlosRuiz>: I just installed this draft to show you the idea
<CarlosRuiz>: but I think to be usable I need some more fixtures
<CarlosRuiz>: and some sample test cases
<CarlosRuiz>: so - maybe in 1 week I guess I can present you something better
<CarlosRuiz>: [ sorry Allan - there was big confusion with the time because DST change ]
<Kai_>: I am looking forward to it. This project looks very promising.
<karsten-thiemann>: I found this case study (for the german readers..) http://www.gebit.de/aktuelles/infomaterial/modellbasiertes_testen0208.pdf
<JoelS>: ok. Allan, we'll post the log for you
<JoelS>: so you can seen the full thread
<CarlosRuiz>: yep - the meetings in #adempiere-team are always logged and published
<JoelS>: ok, thanks for the walk through carlos. we're looking forward to being able to start making the tests
<CarlosRuiz>: sure - I really hope to hear your comments and suggestions in forums - now that the concept is hopefully clear
<CarlosRuiz>: Karsten link translated by google -> http://translate.google.com/translate?prev=_t&hl=en&ie=UTF-8&u=http%3A%2F%2Fwww.gebit.de%2Faktuelles%2Finfomaterial%2Fmodellbasiertes_testen0208.pdf&sl=de&tl=en&history_state0=
<CarlosRuiz>: well - thanks to everybody for attending - more news soon
<Kai_>: OK guys, thank you for everything and good night. :-)
<CarlosRuiz>: and hopefully a better time coordination next time - we keep learning  :-D
<Michael_Judd>: thanks for the effort setting this up joel and carlos !
<Michael_Judd>: well done at cebit Kai and Karsen
<karsten-thiemann>: cebit was fun :)
<Kai_>: yes it was :-)
<karsten-thiemann>: but I was there only for one day - it was teamwork of the whole german adempiere e.v.
<JoelS>: thanks everybody- bye
<karsten-thiemann>: good night everybody
<Kai_>: Good night