Importers

From ADempiere
Revision as of 06:23, 20 August 2010 by CarlosRuiz (Talk) (new category)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search
This Wiki is read-only for reference purposes to avoid broken links.

This page is related to bug with artifact ID 2915713 titled "Importers ignoring exceptions".

This page shall help discuss and fix this bug.

Bug details

Most of importers have a big bad practice logic ignoring exceptions.

Some of them just report the exception and continue, like ImportPayment

catch(Exception e) { log.log(Level.SEVERE, sql.toString(), e); }

Other are even worst like ImportProduct catch (SQLException e) { }

We need a deep review on correct behavior - some exceptions must break the process completely, others can be checked and logged.

This is priority 9 because it can lead to data errors as the import proceed importing some records and others don't, but potentially don't notify the user and/or update the imported record.

Regards,

Carlos Ruiz

Importers

The import processes are in

package org.compiere.process

ImportAccount

Purpose

  • Import Accounts from I_ElementValue

Parameters

  • AD_Client_ID
  • C_Element_ID
  • UpdateDefaultAccounts
  • CreateNewCombination
  • DeleteOldImported

Structure and Relevant Code

//	Delete Old Imported
//	Set Client, Org, IsActive, Created/Updated
//	Set Element
//
//
//	No Name, Value
//	Set Column
//
//	Set Post* Defaults (ignore errors)
//	Summary
//	Doc Controlled
//	Check Account Type A (E) L M O R
//
//	Check Account Sign (N) C B
//
//	No Value
//	****	Update ElementValue from existing
//	-------------------------------------------------------------------
//	Go through Records
		try
		{
//	****	Create/Update ElementValue
		}
		catch (SQLException e)
		{
			throw new Exception ("create", e);
		}
//	Set Error to indicator to not imported
//	*****	Set Parent
		try
		{
		}
		catch (SQLException e)
		{
			log.log(Level.SEVERE, "(ParentUpdateLoop) " + sql.toString(), e);
		}
//	Reset Processing Flag
//	Update Description
//	Done

ImportBankStatement

Purpose

  • Import Bank Statement from I_BankStatement

Parameters

  • AD_Client_ID
  • AD_Org_ID
  • C_BankAccount_ID
  • DeleteOldImported


Structure and Relevant Code

//	Delete Old Imported
//	Set Client, Org, IsActive, Created/Updated
//	Set Bank Account
//
//
//	Set Currency
//
//
//	Set Amount
//
//
//	Set Valuta Date
//	Check Payment<->Invoice combination
//	Check Payment<->BPartner combination
//	Check Invoice<->BPartner combination
//	Check Invoice.BPartner<->Payment.BPartner combination
//	Detect Duplicates
		try
		{
		}
		catch(Exception e)
		{
			log.log(Level.SEVERE, "DetectDuplicates " + e.getMessage());
		}
//Import Bank Statement
		try
		{
		}
		catch(Exception e)
		{
			log.log(Level.SEVERE, sql.toString(), e);
		}
		
//	Set Error to indicator to not imported

ImportBPartner

Purpose

  • Import BPartners from I_BPartner

Parameters

  • AD_Client_ID
  • DeleteOldImported
  • IsValidateOnly


Structure and Relevant Code

//	Delete Old Imported
//	Set Client, Org, IsActive, Created/Updated
//	Set BP_Group
//
//
//	Set Country
//
//
//	Set Region
//	Set Greeting
//
//
//	Existing User ?
//	Existing BPartner ? Match Value
//	Existing Contact ? Match Name
//		Existing Location ? Exact Match
//	Interest Area
// Value is mandatory error
//	Go through Records
		try
		{
//	****	Create/Update BPartner	****
//	****	Create/Update BPartner Location	****
//	****	Create/Update Contact	****
//	Interest Area
		}
		catch (SQLException e)
		{
			rollback();
			//log.log(Level.SEVERE, "", e);
			throw new DBException(e, sql.toString());
		}
		finally
		{
			DB.close(rs, pstmt);
			rs = null; pstmt = null;
			//	Set Error to indicator to not imported
			sql = new StringBuffer ("UPDATE I_BPartner "
					+ "SET I_IsImported='N', Updated=SysDate "
					+ "WHERE I_IsImported<>'Y'").append(clientCheck);
			no = DB.executeUpdateEx(sql.toString(), get_TrxName());
			addLog (0, null, new BigDecimal (no), "@Errors@");
			addLog (0, null, new BigDecimal (noInsert), "@C_BPartner_ID@: @Inserted@");
			addLog (0, null, new BigDecimal (noUpdate), "@C_BPartner_ID@: @Updated@");
		}

ImportConversionRate

Purpose

  • Import Currency Conversion Rates

Parameters

  • AD_Client_ID
  • AD_Org_ID
  • C_ConversionType_ID
  • ValidFrom
  • CreateReciprocalRate
  • DeleteOldImported


Structure and Relevant Code

//	Delete Old Imported
//	Set Client, Org, Location, IsActive, Created/Updated
//	Org
//	Conversion Type
//	Currency
//	Currency To
//	Rates
		try
		{
                // loop
		}
		catch (Exception e)
		{
			log.log(Level.SEVERE, sql.toString(), e);
		}
		try
		{
			if (pstmt != null)
				pstmt.close();
			pstmt = null;
		}
		catch (Exception e)
		{
			pstmt = null;
		}
//	Set Error to indicator to not imported

ImpportGLJournal

Purpose

  • Import GL Journal Batch/JournalLine from I_Journal

Parameters

  • AD_Client_ID
  • AD_Org_ID
  • C_AcctSchema_ID
  • DateAcct
  • IsValidateOnly
  • IsImportOnlyNoErrors
  • DeleteOldImported


Structure and Relevant Code

//	Delete Old Imported
//	Set IsActive, Created/Updated
//	Set Client from Name
//	Set Default Client, Doc Org, AcctSchema, DatAcct
//	Error Doc Org
//	Set AcctSchema
//	Error AcctSchema
//	Set DateAcct (mandatory)
//	Document Type
//	GL Category
//	Set Currency
//	Set Conversion Type
//	Set/Overwrite Home Currency Rate
//	Set Currency Rate
//	Set Period
//	Posting Type
//	** Account Elements (optional) **
//	Set Org from Name (* is overwritten and default)
//	Error Org
//	Set Account
//	Set BPartner
//	Set Product
//	Set Project
//	Set TrxOrg
//	Source Amounts
//	Accounted Amounts (Only if No Error)
//	Get Balance
		try
		{
                // loop
		}
		catch (SQLException ex)
		{
			log.log(Level.SEVERE, sql.toString(), ex);
		}
		try
		{
			if (pstmt != null)
				pstmt.close ();
		}
		catch (SQLException ex1)
		{
		}
		commitEx();
//	Count Errors
		int errors = DB.getSQLValue(get_TrxName(), 
			"SELECT COUNT(*) FROM I_GLJournal WHERE I_IsImported NOT IN ('Y','N')" + clientCheck);

		if (errors != 0)
		{
			if (m_IsValidateOnly || m_IsImportOnlyNoErrors)
				throw new Exception ("@Errors@=" + errors);
		}
		else if (m_IsValidateOnly)
			return "@Errors@=" + errors;
//	Go through Journal Records
		try
		{
                // loop
		}
		catch (Exception e)
		{
			log.log(Level.SEVERE, "", e);
		}
		//	clean up
		try
		{
			if (pstmt != null)
				pstmt.close ();
		}
		catch (SQLException ex1)
		{
		}

ImportInOutConfirm

Purpose

  • Import Confirmations

Parameters

  • AD_Client_ID
  • DeleteOldImported


Structure and Relevant Code

//	Delete Old Imported
//	Set IsActive, Created/Updated
//	Set Client from Name
//	Error Confirmation Line
//	Error Confirmation No
//	Qty
		commitEx();
/*********************************************************************/
		try
		{
                // loop
		}
		catch (Exception e)
		{
			log.log(Level.SEVERE, sql.toString(), e);
		}
		try
		{
			if (pstmt != null)
				pstmt.close ();
			pstmt = null;
		}
		catch (Exception e)
		{
			pstmt = null;
		}

ImportInventory

Purpose

  • Import Physical Inventory from I_Inventory

Parameters

  • AD_Client_ID
  • AD_Org_ID
  • M_Locator_ID
  • MovementDate
  • DeleteOldImported
  • IsUpdateCosting
  • C_AcctSchema_ID
  • M_CostType_ID
  • M_CostElement_ID
  • AD_OrgTrx_ID

Structure and Relevant Code

//	Delete Old Imported
//	Set Client, Org, Location, IsActive, Created/Updated
//	Location
//	Set M_Warehouse_ID
//	Product
//	No QtyCount
commitEx();
//	Go through Inventory Records
		try
		{
// MInventory
// MAttributeSet
// MInventoryLine
		}
		catch (Exception e)
		{
			log.log(Level.SEVERE, sql.toString(), e);
		}

ImportInvoice

ImportOrder

ImportPayment

ImportProduct

ImportReportLine

Related Process

ImportDelete

Purpose

Parameters

Structure and Relevant Code


Fixing the bug

The collection of exceptions to be thrown where applicable (besides java.lang.Exception) are in

package org.adempiere.exceptions

  • AdempiereException
    Any exception that occurs inside the Adempiere core
  • DBException
    This RuntimeException is used to pass SQLException up the chain of calling
    methods to determine what to do where needed.

package org.compiere.util

  • Adempiere User Error.
    Cuased by (lack of) user input/selection.
    (No program error)
  • Adempiere System Error.
    Error caused by invalid configurations, etc.
    (No program error)