Latest revision as of 05:39, 31 May 2011

Note: This work contains contributions from Victor Perez/e-Evolution, Carlos Ruiz/GlobalQSS, InterOpen/China, Sysnova, Redhuan D. Oon and others. This Testing is for the new Cost Engine. You may download a complete guide here.

Fig.1 Cost Engine Test Suite based on FitnesseSlim, designed as a user-friendly modular Suite of StoryTests that you can run in different sets.

Introduction

Fig.2 - Setup is the common unit used by all StoryTests. It contains Login parameters. You can run any storytest or whole suite and they will always include this Setup for you.

Fig.3 You can run Generate Cost Transaction as Suite or as a standalone StoryTest. All column values are editable in the wiki.

Fig.4 - This StoryTest ran successfully on the product defined in Setup. If you expand the included Setup you will see the other 2 right.

• Testing is vital and crucial as software death can happen.
• We need to examine the overall compliance to our best practice besides going through the features, requirements and test plan.
• Then we examine the best tools for a test suite.
  • Fitnesse is one such tool already explored by Carlos Ruiz and Ivan Calderon's team at Interopen.
    • Interopen has made PMC:QA page and setup with Hudson.
  • Fitnesse expands the concept of testing to be more bazaar like and allow anyone out there to test easily, diversely and constantly.
  • It is easily upgraded to SLIM, a more powerful version of Fitnesse. (See right)

Scope Deliberation

To come to an understanding of a scope, it is best to grasp where we are and where we want to go.

General Proposition of an ERP

• The essence of a modern ERP today lies in its Financials Integration.
• The core of that is the Costing Engine that fulfills advanced Cost Accounting requirements.
• A Costing Engine is a highly challenging module to develop and even SAP cannot perfectly resolve all scenarios.
• ADempiere seeks to resolve the most common terms of advanced use, such as Average Costing with Average Invoicing, FiFO and LiFO.

State of Affairs

• Compiere, which ADempiere inherits from does not have a complete costing method beyond Standard Costing.
• In developing methods like Average Costing, e-Evolution has proposed that a complete writing of it is necessary.
• Thus the Sponsored Development: Libero Cost Engine is born.

Constraints Environment

• First we be asking questions such as:
• Will the Web Ui be tested? Or is the Costing Engine merely confined to itself and not its performance over the web?
• What will be the maximum type of Organisations encountered?
  • There can be about 4 basic organisations to represent each costing method:

1. OrgStd - Organisation that uses Standard Costing
2. OrgAve - Organisation that uses Average Costing
3. OrgFif - Organisation that uses First In First Out Costing
4. OrgLif - Organisation that uses Last In First Out Costing

• When are the other branches or vertical projects that have impact or vice versa?
• What are the parts that we will NOT be testing?
• What will comprise a successful testing and how will the results be presented or documented?
• Who can be the independent testers and what tools can they use?
• How will testers use the same tools and deal with different set of data or results?
• What will be weakness or caveat of the tools used?

Dictionary or Terms

Using QUERY Table for Accounts Setup to extract accounting consequence during transaction testing.

• Costing Method - Standard, Average, FIFO, LIFO.

Index of Discussion Threads

(Listed in reverse chronology order - latest first)

Average Costing

• Mike Judd 2010-08-14
• Marek 2010-06-25
• Azzam Ahmad 2010-05-16
• Sanyasi 2008-02-13
• Armen Rizal 2007-06-15
• Joel S 2007-01-11

Standard Costing

• Bayu 2008-11-27

Test Plan

Testing Framework Toolset

• The TestFramework to be used is Slim Fitnesse.
• Decision Tables will send sample data and assertions to a Fixture that describes a Use Case that access a JUnit class testing the Cost Engine.
• The Decision Tables are wiki-like where any user can edit the sample data.
• Tests done in Fitnesse do not change, add or delete any records in the Database. Any transactions will not persist or commit in the DB even if you abort the test run halfway.
• Fitnesse can be run as a Suite to share common Setup and its child tests can be turned off individually.
• This allows maximum reuse, visibility, high readability and scenario building without editing JUnit classes.

Environment Setup

• As shown in above screen, !paths are set to the latest Adempiere.jar and AdempiereClib.jar which was RUN_Setup with the LiberoCE.jar (renamed to customization.jar) and patches.jar in the ADempiereHome/lib folder.
  • Login to System and GardenAdmin to Run Role Access Update after that and you are ready to start the tests.
• Other !paths are for Fitnesse/bin workspace which testing code is been developed and run, and fitnesse.jar in use.

Specifications and Document Review

POC of first row of test, Purchase and Receipt for 10 Oaks at $36. 2nd row is a Sales or 5 Oaks, and so on. Hard asserts are not used but dynamic output in grey are helpful enough.

Here is a completely asserted Test after Cost Engine code correction, thanks to the last test. The Ref column refers to previous associated transactions.

This part answer the question whether there exist sufficient documentation of specifications for proper review and whether the codings are in accordance to ADempiere Best Practices.

• Community Involvement - Yes
  • Discussion Thread - Yes
  • Critical Need - Yes /as Feature /accounts impact
  • Overlap with other branches - Yes: Armen, Carlos (according to discussion threads)
  • Market Comparison - Existing standard
• Specifications Document - Yes, still pending review
  • Versioning of Document - Not Visible, still pending review
  • Testing Document - Not Visible, still pending review
  • Review and Feedback - Not Entirely, still pending review
• Code Practice Compliance
  • In-code documentation - pending review
  • Code naming convention - Yes, still pending review
  • Follow patterns - pending review
  • Commit to Branch - Yes, still pending review
  • Backward compatible - Core code changes /core model changes
  • Impact to feature - Standard Costing
  • Reference to Document Specs - Not Specific

Use Cases

After more test rows, some errors begin to show. This allows quick cycle of develop, test, amend during early stages.

General Description

Testing Criteria:Product Assets = Inventory Valuation Report ( except decimals by divide & multiply)

• Account Info ( Product Assets & Each product ) = Transactions Valuation Report

= Statement of Accounts ( for Each Product )

• For items purchased :Total purchasing amounts compared to total cost of goods sold.

Testing Cases :

- Items without Landed Cost

- Items With Cost

- Foreign Currency Purchasing

- Transactions on products with zero cost

• Testing Transactions ( Windows ):

- Material Receipt

- Shipment

- Internal Use Inventory

- Physical Inventory.

- Inventory Move.

- Customer RMA

- Vendor RMA

Case Index

• Case I : Single Organization with Seed DB (adapting Victor's JUnit test)
• Case II: Landed Cost with backdated transaction (from UserGuide Exercise)
• Case III: Multiple Organisations with different Costing Methods.
• Case IV: Different currency purchase
• Case V: Sales with Negative Inventory
• Case VI: RMA
• Manufacturing finished goods cost with Average Cost of raw materials (not valid)
• Case VIII: Manufacturing Standard Costing
• Case IX: Mandatory Attribute Set Instance
• Case X: Counter Document for PO to SO in separate linked Orgs

Black Box Testing

• Description of Use Cases that are common complete and quite comprehensive business scenario. Good example is Accounting Consequence where Accounts Posting have been remarkably successful in the testings.

Conditional Testing

• Specific tests that are white-box. They look into the code and ensure a certain loop condition is exhausted.

Issues Reference

• (link)

Test Suite

Overview of Logic

• Methods are called from Fitnesse Testing Framework
• Each Method execute a DocType Transaction with parameters such as DocType, Qty, and Price.
• At end of Method, MCost and CostDetail values are retrieved for assert at the Fitnesse page.
• Please look at Cost Engine/Case I as working example.

RoadMap as of Feb 1 2011

Consecutive tasks based on working days:

• Accounting Consequence capture during test - 1 day. POC Done and Proven - 03:44, 5 February 2011 (UTC)
• Accounts Posting Fixture - 2 days. DONE, COMMITTED - 07:35, 6 February 2011 (UTC)
• Case II - 3 days - DONE
• Case III - 3 days - DONE (Issues pending)
• Case IX - DONE
• Case X - DONE
• Case V - DONE - 13:29, 28 February 2011 (UTC)
• Update of Online wiki pages - 2 days - ONGOING
• Update of PDF Guide to version 0.3 - Not needed
• CASE IV - VIII - 2 weeks

You are invited

• If you wish to join in please sign with (*~~~) below.
• Redhuan D. Oon :)

Links

• Trackers submitted by Redhuan D. Oon that reports and resolves ongoing development issues with the Cost Engine.
• Fitnesse Slim POC committed to Trunk
• Introducing this page to SourceForge Forum.
• Sponsored Development: Libero Cost Engine
• Guide on how to setup the testing engine
• Learn from the start at TestFramework with enhancement notes in FitnesseSlim.