Difference between revisions of "Import Validation Helper"

From ADempiere
Jump to: navigation, search
This Wiki is read-only for reference purposes to avoid broken links.
m (Problem 2: Cumbersome in writing a lot of validation routine in Import Process)
m (Problem 2: Cumbersome in writing a lot of validation routine in Import Process)
Line 42: Line 42:
 
</pre>
 
</pre>
  
And one fact it, a big portion of code in import process, are dedicated for validation.
+
A big portion of code in import process, are dedicated for validation.
  
 
If using the existing Import Window that already done with ADempiere, it is fine. But when it come to create my own import window, or adding more fields to validate, it is time consuming and error prone.
 
If using the existing Import Window that already done with ADempiere, it is fine. But when it come to create my own import window, or adding more fields to validate, it is time consuming and error prone.

Revision as of 01:27, 9 November 2011

Overview of Data Import Validation

In ADempiere, when come to importing data from legacy system, normally we turn to Data Import module (&Menu > System Admin > Data > Data Import).

Following picture depict the concept of how this module works.

Data import concept.png

If you have been working, and digging into this module before, you will know that the most important part is the validation of raw data --> to data that is valid to ADempiere.

For example, when importing Product, one of the data required is "Product Category". In the Import Product window, you will see 2 fields.

Import product sample.png

When run Import Product, the "Product Category Key" will be validated (using SQL) to get a valid M_Product_Category_ID for "Product Category".

i.e.,

Product Category Key = "TILE" --> Product Category = "Tile" (M_Product_Category_ID = 1000001).

All these looks OK, it work as it should be, but what is my problem?

My personal problem about Data Import in ADempiere

Reasons that I come up with this Import Validation Helper is due to some of the following problem I face,

Problem 1: Incompatible source of data

For that case, that you have control over the source data, it is fine. We can create that CSV file with the valid "Key". But I my case, I need to reuse product data (CSV file) from other system,

  1. Which may not use the same category key as Adempiere, i.e., they use "TL" --> I need to map it to "TILE" before upload to Import Product table.
  2. Which may need to regrouping, i.e., their group "TL", "TT", "TB" --> I need to map all of them to "TILE" group in ADempiere.

And this can't be done automatically in ADempiere's Import Process.

Problem 2: Cumbersome in writing a lot of validation routine in Import Process

If you look in to the code, i.e., ImportProduct.java, you will notice that the first portion of code (in doIt() function) is dedicated for Validation. I.e., for validate M_Product_Category_ID, we have

		sql = new StringBuffer ("UPDATE I_Product i "
			+ "SET M_Product_Category_ID=(SELECT M_Product_Category_ID FROM M_Product_Category c"
			+ " WHERE i.ProductCategory_Value=c.Value AND i.AD_Client_ID=c.AD_Client_ID) "
			+ "WHERE ProductCategory_Value IS NOT NULL AND M_Product_Category_ID IS NULL"
			+ " AND I_IsImported<>'Y'").append(clientCheck);
		no = DB.executeUpdate(sql.toString(), get_TrxName());
		log.info("Set Category=" + no);

A big portion of code in import process, are dedicated for validation.

If using the existing Import Window that already done with ADempiere, it is fine. But when it come to create my own import window, or adding more fields to validate, it is time consuming and error prone.

Business Requirement

How to install this plugin

Step by Step Guide