Difference between revisions of "Cost Engine/Testing"

From ADempiere
Jump to: navigation, search
This Wiki is read-only for reference purposes to avoid broken links.
(2nd row done!)
m (→‎Case Index: link to)
 
(42 intermediate revisions by the same user not shown)
Line 1: Line 1:
−
[[Image:FitnesseCostEngineSuite.gif|thumb|440px|right|Cost Engine Test Suite based on [[FitnesseSlim|Slim]] version of [[Fitnesse]], designed as a ''Suite'' of ''Column Tables''.]]
+
{{Note|This work contains contributions from Victor Perez/e-Evolution, Carlos Ruiz/GlobalQSS, InterOpen/China, Sysnova, [[User:Red1|Redhuan D. Oon]] and others. This Testing is for the [[Sponsored Development: Libero Cost Engine|new Cost Engine]]. You may download a [http://sourceforge.net/projects/adempiere/files/Adempiere%20Packages/Libero%20Costing%20Engine%20Test/CostEngineTesting.pdf/download complete guide here]. }}
 +
 
 +
[[Image:FitnesseCostEngineSuite.gif|thumb|400px|right|Fig.1 Cost Engine Test Suite based on [[FitnesseSlim]], designed as a user-friendly modular ''Suite'' of ''StoryTests'' that you can run in different sets.]]
  
 
=Introduction=
 
=Introduction=
−
[[Image:FitnesseCostingEngineAssertRows.gif|thumb|440px|right|The ''Column Table'' for iterating each transaction and asserting their costing results.]]  
+
[[Image:FitnesseCostEngineSetup.gif|thumb|400px|right|Fig.2 - '''Setup''' is the common unit used by all ''StoryTests''. It contains Login parameters. You can run any storytest or whole suite and they will always include this '''Setup''' for you.]]
 +
[[Image:FitnesseCostEngineGenerateCostTransaction.gif|thumb|400px|right|Fig.3 You can run '''Generate Cost Transaction''' as ''Suite'' or as a standalone ''StoryTest''. All column values are editable in the wiki.]]
 +
[[Image:FitnesseCostEngineGenerateCostTransactionSuccess.gif|thumb|400px|right|Fig.4 - This StoryTest ran successfully on the product defined in Setup. If you expand the included ''Setup'' you will see the other 2 ''right''.]]  
 +
 
 
*Testing is vital and crucial as [http://fitnesse.org/FitNesse.UserGuide.ProjectDeathByRequirements software death] can happen.
 
*Testing is vital and crucial as [http://fitnesse.org/FitNesse.UserGuide.ProjectDeathByRequirements software death] can happen.
 
*We need to examine the overall compliance to our best practice besides going through the features, requirements and test plan.
 
*We need to examine the overall compliance to our best practice besides going through the features, requirements and test plan.
Line 9: Line 14:
 
***Interopen has made [[PMC:QA]] page and setup with [http://www.adempiereqa.com/hudson/job/Fitnesse/ Hudson].
 
***Interopen has made [[PMC:QA]] page and setup with [http://www.adempiereqa.com/hudson/job/Fitnesse/ Hudson].
 
**Fitnesse expands the concept of testing to be more bazaar like and allow anyone out there to test easily, diversely and constantly.
 
**Fitnesse expands the concept of testing to be more bazaar like and allow anyone out there to test easily, diversely and constantly.
−
**It is easily upgraded to SLIM, a more powerful version of Fitnesse. (See right)
+
**It is easily upgraded to [[FitnesseSlim|SLIM]], a more powerful version of [[Fitnesse]]. (See right)
−
**Fitnesse can be run as a ''Suite'' where reusable statements are ''Setup'' appearing on a single page.
+
  
 
=Scope Deliberation=
 
=Scope Deliberation=
Line 41: Line 45:
  
 
==Dictionary or Terms==
 
==Dictionary or Terms==
 +
[[Image:FitnesseAcctsPostSetup.gif|thumb|right|400px|Using QUERY Table for Accounts Setup to extract accounting consequence during transaction testing.]]
 
*Costing Method - Standard, Average, FIFO, LIFO.
 
*Costing Method - Standard, Average, FIFO, LIFO.
  
Line 57: Line 62:
 
=Test Plan=
 
=Test Plan=
 
==Testing Framework Toolset==  
 
==Testing Framework Toolset==  
−
[[Image:FitnesseCostEngineTest1.gif|thumb|right|440px|First stage testing to setup Historical Data, asserting successful generation of Cost Transaction]]
 
−
 
 
*The [[TestFramework]] to be used is ''[http://fitnesse.org/FitNesse.UserGuide.FixtureCode Slim Fitnesse]''.
 
*The [[TestFramework]] to be used is ''[http://fitnesse.org/FitNesse.UserGuide.FixtureCode Slim Fitnesse]''.
 
*''Decision Tables'' will send sample data and assertions to a ''Fixture'' that describes a ''Use Case'' that access a ''JUnit'' class testing the ''Cost Engine''.
 
*''Decision Tables'' will send sample data and assertions to a ''Fixture'' that describes a ''Use Case'' that access a ''JUnit'' class testing the ''Cost Engine''.
 
*The ''Decision Tables'' are ''wiki-like'' where any user can edit the sample data.
 
*The ''Decision Tables'' are ''wiki-like'' where any user can edit the sample data.
 +
*Tests done in Fitnesse do not change, add or delete any records in the Database. Any transactions will not persist or commit in the DB even if you abort the test run halfway.
 +
*Fitnesse can be run as a ''Suite'' to share common ''Setup'' and its child tests can be turned off individually.
 
*This allows maximum reuse, visibility, high readability and scenario building without editing JUnit classes.
 
*This allows maximum reuse, visibility, high readability and scenario building without editing JUnit classes.
 
===Environment Setup===
 
===Environment Setup===
Line 69: Line 74:
  
 
==Specifications and Document Review==
 
==Specifications and Document Review==
−
[[Image:FitnesseCostEngineResults.gif|thumb|500px|right|POC of first row of test, Purchase and Receipt for 10 Oaks at $36. 2nd row is a Sales or 5 Oaks. Hard asserts are not used but dynamic output in grey are helpful enough.]]
+
[[Image:FitnesseCostEngineResults.gif|thumb|500px|right|POC of first row of test, Purchase and Receipt for 10 Oaks at $36. 2nd row is a Sales or 5 Oaks, and so on. Hard asserts are not used but dynamic output in grey are helpful enough.]]
−
''Information gathering in progress. Please do not edit - [[User:Red1|Redhuan D. Oon]]''
+
[[Image:FitnesseCostingTransactionText.gif|thumb|500px|right|Here is a completely asserted Test after Cost Engine code correction, thanks to the last test. The Ref column refers to previous associated transactions.]]
 
This part answer the question whether there exist sufficient documentation of specifications for proper review and whether the codings are in accordance to [[ADempiere Best Practices]].
 
This part answer the question whether there exist sufficient documentation of specifications for proper review and whether the codings are in accordance to [[ADempiere Best Practices]].
 
*Community Involvement - Yes
 
*Community Involvement - Yes
Line 91: Line 96:
  
 
==Use Cases==
 
==Use Cases==
−
*Case I : Single Organization with Seed DB
+
[[Image:FitnesseTestError.gif|thumb|right|500px|After more test rows, some errors begin to show. This allows quick cycle of develop, test, amend during early stages.]]
−
*Case II: Single Organization with data in live environment.
+
===General Description===
−
*Case III
+
−
*Case IV
+
−
*Case V
+
−
* ...
+
−
 
+
 
Testing Criteria:Product Assets = Inventory Valuation Report ( except decimals by divide & multiply)
 
Testing Criteria:Product Assets = Inventory Valuation Report ( except decimals by divide & multiply)
−
<pre>Account Info ( Product Assets & Each product ) = Transactions Valuation Report  
+
*Account Info ( Product Assets & Each product ) = Transactions Valuation Report  
−
    = Statement of Accounts ( for Each Product )
+
:= Statement of Accounts ( for Each Product )
−
For items purchased :Total purchasing amounts compared to total cost of goods sold.
+
*For items purchased :Total purchasing amounts compared to total cost of goods sold.
−
Testing Cases :
+
:Testing Cases :
−
+
::- Items without Landed Cost
−
          - Items without Landed Cost
+
::- Items With Cost
−
          - Items With Cost
+
::- Foreign Currency Purchasing
−
          - Foreign Currency Purchasing
+
::- Transactions on products with zero cost
−
          - Transactions on products with zero cost
+
  
−
Testing Transactions ( Windows ):
+
*Testing Transactions ( Windows ):
 +
::- Material Receipt
 +
::- Shipment
 +
::- Internal Use Inventory
 +
::- Physical Inventory.
 +
::- Inventory Move.
 +
::- Customer RMA
 +
::- Vendor RMA
  
−
          - Material Receipt
+
===Case Index===
−
          - Shipment
+
*[[Cost Engine/Case I|Case I]] : Single Organization with Seed DB (adapting Victor's JUnit test)
−
          - Internal Use Inventory
+
*[[Cost Engine/Case II|Case II]]: Landed Cost with backdated transaction (from UserGuide Exercise)
−
          - Physical Inventory.
+
*[[Cost Engine/Case III|Case III]]: Multiple Organisations with different Costing Methods.
−
          - Inventory Move.
+
*[[Cost Engine/Case IV|Case IV]]: Different currency purchase
−
          - Customer RMA
+
*[[Cost Engine/Case V|Case V]]: Sales with Negative Inventory
−
          - Vendor RMA
+
*[[Cost Engine/Case VI|Case VI]]: RMA
−
</pre>
+
* Manufacturing finished goods cost with Average Cost of raw materials (not valid)
 +
*[[Manufacturing/Case I|Case VIII]]: Manufacturing Standard Costing
 +
*[[Cost Engine/Case IX|Case IX]]: Mandatory Attribute Set Instance
 +
*[[Cost Engine/Case X|Case X]]: Counter Document for PO to SO in separate linked Orgs
  
 
===Black Box Testing===
 
===Black Box Testing===
−
*Description of Use Cases that are common complete and quite comprehensive business scenario
+
*Description of Use Cases that are common complete and quite comprehensive business scenario. Good example is ''Accounting Consequence'' where '''Accounts Posting''' have been remarkably successful in the testings.
  
 
===Conditional Testing===
 
===Conditional Testing===
Line 130: Line 139:
  
 
=Test Suite=
 
=Test Suite=
 +
==Overview of Logic==
 +
*Methods are called from Fitnesse Testing Framework
 +
*Each Method execute a DocType Transaction with parameters such as DocType, Qty, and Price.
 +
*At end of Method, MCost and CostDetail values are retrieved for assert at the Fitnesse page.
 +
*Please look at [[Cost Engine/Case I]] as working example.
 +
 +
=RoadMap as of Feb 1 2011=
 +
Consecutive tasks based on working days:
 +
*Accounting Consequence capture during test - 1 day. POC Done and Proven -  03:44, 5 February 2011 (UTC)
 +
*Accounts Posting Fixture - 2 days. DONE, COMMITTED - 07:35, 6 February 2011 (UTC)
 +
*[[Cost Engine/Case II|Case II]] - 3 days - DONE
 +
*[[Cost Engine/Case III|Case III]] - 3 days - DONE (Issues pending)
 +
*[[Cost Engine/Case IX|Case IX]] - DONE
 +
*[[Cost Engine/Case X|Case X]] - DONE
 +
*[[Cost Engine/Case V|Case V]] - DONE - 13:29, 28 February 2011 (UTC)
 +
*Update of Online wiki pages - 2 days - ONGOING
 +
*Update of PDF Guide to version 0.3 - Not needed
 +
*CASE IV - VIII - 2 weeks
  
 
=You are invited=
 
=You are invited=
Line 136: Line 163:
  
 
=Links=
 
=Links=
−
*Start with [[TestFramework]] with enhancement notes in [[FitnesseSlim]].
+
*[http://sourceforge.net/tracker/?limit=25&func=&group_id=176962&atid=1349578&assignee=&status=&category=&artgroup=&keyword=&submitter=&artifact_id=&assignee=&status=&category=&artgroup=&submitter=red1&keyword=&artifact_id=&submit=Filter&mass_category=&mass_priority=&mass_resolution=&mass_assignee=&mass_artgroup=&mass_status=&mass_cannedresponse= Trackers submitted] by [[User:Red1|Redhuan D. Oon]] that reports and resolves ongoing development issues with the Cost Engine.
−
*[http://adempiere.svn.sourceforge.net/viewvc/adempiere/FitnesseSlim/ Fitnesse Slim] POC committed to Trunk
+
*[http://adempiere.svn.sourceforge.net/viewvc/adempiere/branches/FitnesseSlim/ Fitnesse Slim] POC committed to Trunk
 
*Introducing this page to [http://sourceforge.net/projects/adempiere/forums/forum/611167/topic/4052780 SourceForge Forum].
 
*Introducing this page to [http://sourceforge.net/projects/adempiere/forums/forum/611167/topic/4052780 SourceForge Forum].
 
*[[Sponsored Development: Libero Cost Engine]]
 
*[[Sponsored Development: Libero Cost Engine]]
 +
*[[Cost_Engine/Case_I#How_To_Use|Guide]] on how to setup the testing engine
 +
*Learn from the start at [[TestFramework]] with enhancement notes in [[FitnesseSlim]].
  
 
[[Category:LIBERO]]
 
[[Category:LIBERO]]

Latest revision as of 05:39, 31 May 2011

Note.gif Note:

This work contains contributions from Victor Perez/e-Evolution, Carlos Ruiz/GlobalQSS, InterOpen/China, Sysnova, Redhuan D. Oon and others. This Testing is for the new Cost Engine. You may download a complete guide here.

Fig.1 Cost Engine Test Suite based on FitnesseSlim, designed as a user-friendly modular Suite of StoryTests that you can run in different sets.

Introduction

Fig.2 - Setup is the common unit used by all StoryTests. It contains Login parameters. You can run any storytest or whole suite and they will always include this Setup for you.
Fig.3 You can run Generate Cost Transaction as Suite or as a standalone StoryTest. All column values are editable in the wiki.
Fig.4 - This StoryTest ran successfully on the product defined in Setup. If you expand the included Setup you will see the other 2 right.
  • Testing is vital and crucial as software death can happen.
  • We need to examine the overall compliance to our best practice besides going through the features, requirements and test plan.
  • Then we examine the best tools for a test suite.
    • Fitnesse is one such tool already explored by Carlos Ruiz and Ivan Calderon's team at Interopen.
    • Fitnesse expands the concept of testing to be more bazaar like and allow anyone out there to test easily, diversely and constantly.
    • It is easily upgraded to SLIM, a more powerful version of Fitnesse. (See right)

Scope Deliberation

To come to an understanding of a scope, it is best to grasp where we are and where we want to go.

General Proposition of an ERP

  • The essence of a modern ERP today lies in its Financials Integration.
  • The core of that is the Costing Engine that fulfills advanced Cost Accounting requirements.
  • A Costing Engine is a highly challenging module to develop and even SAP cannot perfectly resolve all scenarios.
  • ADempiere seeks to resolve the most common terms of advanced use, such as Average Costing with Average Invoicing, FiFO and LiFO.

State of Affairs

Constraints Environment

  • First we be asking questions such as:
  • Will the Web Ui be tested? Or is the Costing Engine merely confined to itself and not its performance over the web?
  • What will be the maximum type of Organisations encountered?
    • There can be about 4 basic organisations to represent each costing method:
  1. OrgStd - Organisation that uses Standard Costing
  2. OrgAve - Organisation that uses Average Costing
  3. OrgFif - Organisation that uses First In First Out Costing
  4. OrgLif - Organisation that uses Last In First Out Costing
  • When are the other branches or vertical projects that have impact or vice versa?
  • What are the parts that we will NOT be testing?
  • What will comprise a successful testing and how will the results be presented or documented?
  • Who can be the independent testers and what tools can they use?
  • How will testers use the same tools and deal with different set of data or results?
  • What will be weakness or caveat of the tools used?

Dictionary or Terms

Using QUERY Table for Accounts Setup to extract accounting consequence during transaction testing.
  • Costing Method - Standard, Average, FIFO, LIFO.

Index of Discussion Threads

(Listed in reverse chronology order - latest first)

Average Costing

Standard Costing

Test Plan

Testing Framework Toolset

  • The TestFramework to be used is Slim Fitnesse.
  • Decision Tables will send sample data and assertions to a Fixture that describes a Use Case that access a JUnit class testing the Cost Engine.
  • The Decision Tables are wiki-like where any user can edit the sample data.
  • Tests done in Fitnesse do not change, add or delete any records in the Database. Any transactions will not persist or commit in the DB even if you abort the test run halfway.
  • Fitnesse can be run as a Suite to share common Setup and its child tests can be turned off individually.
  • This allows maximum reuse, visibility, high readability and scenario building without editing JUnit classes.

Environment Setup

  • As shown in above screen, !paths are set to the latest Adempiere.jar and AdempiereClib.jar which was RUN_Setup with the LiberoCE.jar (renamed to customization.jar) and patches.jar in the ADempiereHome/lib folder.
    • Login to System and GardenAdmin to Run Role Access Update after that and you are ready to start the tests.
  • Other !paths are for Fitnesse/bin workspace which testing code is been developed and run, and fitnesse.jar in use.

Specifications and Document Review

POC of first row of test, Purchase and Receipt for 10 Oaks at $36. 2nd row is a Sales or 5 Oaks, and so on. Hard asserts are not used but dynamic output in grey are helpful enough.
Here is a completely asserted Test after Cost Engine code correction, thanks to the last test. The Ref column refers to previous associated transactions.

This part answer the question whether there exist sufficient documentation of specifications for proper review and whether the codings are in accordance to ADempiere Best Practices.

  • Community Involvement - Yes
    • Discussion Thread - Yes
    • Critical Need - Yes /as Feature /accounts impact
    • Overlap with other branches - Yes: Armen, Carlos (according to discussion threads)
    • Market Comparison - Existing standard
  • Specifications Document - Yes, still pending review
    • Versioning of Document - Not Visible, still pending review
    • Testing Document - Not Visible, still pending review
    • Review and Feedback - Not Entirely, still pending review
  • Code Practice Compliance
    • In-code documentation - pending review
    • Code naming convention - Yes, still pending review
    • Follow patterns - pending review
    • Commit to Branch - Yes, still pending review
    • Backward compatible - Core code changes /core model changes
    • Impact to feature - Standard Costing
    • Reference to Document Specs - Not Specific

Use Cases

After more test rows, some errors begin to show. This allows quick cycle of develop, test, amend during early stages.

General Description

Testing Criteria:Product Assets = Inventory Valuation Report ( except decimals by divide & multiply)

  • Account Info ( Product Assets & Each product ) = Transactions Valuation Report
= Statement of Accounts ( for Each Product )
  • For items purchased :Total purchasing amounts compared to total cost of goods sold.
Testing Cases :
- Items without Landed Cost
- Items With Cost
- Foreign Currency Purchasing
- Transactions on products with zero cost
  • Testing Transactions ( Windows ):
- Material Receipt
- Shipment
- Internal Use Inventory
- Physical Inventory.
- Inventory Move.
- Customer RMA
- Vendor RMA

Case Index

  • Case I : Single Organization with Seed DB (adapting Victor's JUnit test)
  • Case II: Landed Cost with backdated transaction (from UserGuide Exercise)
  • Case III: Multiple Organisations with different Costing Methods.
  • Case IV: Different currency purchase
  • Case V: Sales with Negative Inventory
  • Case VI: RMA
  • Manufacturing finished goods cost with Average Cost of raw materials (not valid)
  • Case VIII: Manufacturing Standard Costing
  • Case IX: Mandatory Attribute Set Instance
  • Case X: Counter Document for PO to SO in separate linked Orgs

Black Box Testing

  • Description of Use Cases that are common complete and quite comprehensive business scenario. Good example is Accounting Consequence where Accounts Posting have been remarkably successful in the testings.

Conditional Testing

  • Specific tests that are white-box. They look into the code and ensure a certain loop condition is exhausted.

Issues Reference

  • (link)

Test Suite

Overview of Logic

  • Methods are called from Fitnesse Testing Framework
  • Each Method execute a DocType Transaction with parameters such as DocType, Qty, and Price.
  • At end of Method, MCost and CostDetail values are retrieved for assert at the Fitnesse page.
  • Please look at Cost Engine/Case I as working example.

RoadMap as of Feb 1 2011

Consecutive tasks based on working days:

  • Accounting Consequence capture during test - 1 day. POC Done and Proven - 03:44, 5 February 2011 (UTC)
  • Accounts Posting Fixture - 2 days. DONE, COMMITTED - 07:35, 6 February 2011 (UTC)
  • Case II - 3 days - DONE
  • Case III - 3 days - DONE (Issues pending)
  • Case IX - DONE
  • Case X - DONE
  • Case V - DONE - 13:29, 28 February 2011 (UTC)
  • Update of Online wiki pages - 2 days - ONGOING
  • Update of PDF Guide to version 0.3 - Not needed
  • CASE IV - VIII - 2 weeks

You are invited

  • If you wish to join in please sign with (*~~~) below.
  • Redhuan D. Oon :)

Links

Retrieved from "http://www.adempiere.com/index.php5?title=Cost_Engine/Testing&oldid=9425"