This Wiki is read-only for reference purposes to avoid broken links.

Latest revision as of 06:41, 12 November 2012

Chapter 4. Compiling and Extending
Prev

Compiling and Extending

Compiling Migrate

Normally there should be no need to compile Migrate, as it will be installed together with Adempiere.

However, there may be situations when you separately want to compile Migrate, either to modify the code to suit your personal needs, or to fix bugs or extend the code and hopefully contribute your enhancements to the Adempiere project.

Requirements

Migrate requires the Java Development Kit version 1.7 (JDK 7)⁸ and therefore also at least version 3.7.0lts of Adempiere.

Downloading and Compiling the Source Code

Download the Adempiere source.
hg clone http://adempiere.hg.sourceforge.net/hgroot/adempiere/adempiere#development .
You can either compile the complete Adempiere project or only the Migrate sub-project.
- To compile the complete Adempiere project, change to directory utils_dev.
  cd utils_dev
- To compile only the Migrate sub-project, change to directory migrate.
  cd migrate
Then execute RUN_build.sh (or RUN_build.bat).
./RUN_build.sh
The resulting JAR file (migrate.jar) will be created in the migrate project directory and also copied to the ../lib directory.
This will also generate the API and user documentation, to be found in the migrate/apidoc and migrate/userdoc directories, respectively.

For details on how to work with Adempiere source code, consult the Adempieredocumentation.

Building and Running Migrate in Eclipse

Consult the Adempieredocumentation on how to compile and run Adempiere from within ECLIPSE.

Note that the JDBC drivers for installed databases must be in the classpath.

If you have installed Adempiere, they can be found in $ADEMPIERE_HOME/lib:

$ADEMPIERE_HOME/lib/oracle.jar for Oracle
$ADEMPIERE_HOME/lib/postgresql.jar for postgreSQL

Otherwise they can be found in subdirectories of your local database installation, for example

$ORACLE_HOME/jdbc/lib/ojdbc14.jar for Oracle
/usr/share/java/postgresql-jdbc.jar for postgreSQL

To add files or directories to the classpath in Eclipse (version 3.4.1), in the Run menu select Run Configurations…, select the Classpath tab and click the Add External JARs… button.

JDBC drivers must be set in the classpath for Migrate to run in Eclipse

Extending Migrate

Source Files

Being open-source, Migrate has the advantage that you can modify the source code to fit your particular needs.

More than that, Migrate is designed to be easily extendable for localization and for handling additional database vendors, and you are invited to help and contribute your solutions to Adempiere.

To help you navigate the source files, they are listed here by category:

Table 4.1. Source Files

Category Source Files

Main class

Migrate.java

Parameters and constants

Parameters.java

Graphical User Interface

Gui.java

HelpAbout.java

HelpInfo.java

images/*

Logging

MigrateLogger.java

MigrateLogger_Formatter.java

MigrateLogger_Filter.java

PreparedStatementWrapper.java

Localization

Messages.java

User Documentation

manual.xml

images/doc_….png

JDBC connection to database

DBConnection.java

Vendor-specific SQL-generation and database rules and conventions

DBEngine.java

DBEngineInterface.java

DBEngine_Oracle.java

DBEngine_Postgresql.java

Database objects

DBObject.java

DBObjectInterface.java

DBObjectDefinition.java

DBObject_Table.java

DBObject_Table_Column.java

DBObject_PrimaryKey.java

DBObject_PrimaryKey_Table.java

DBObject_PrimaryKey_Column.java

DBObject_ForeignKey.java

DBObject_ForeignKey_Table.java

DBObject_ForeignKey_Column.java

DBObject_Check.java

DBObject_Check_Table.java

DBObject_Check_Rule.java

DBObject_Unique.java

DBObject_Unique_Table.java

DBObject_Unique_Column.java

DBObject_Index.java

DBObject_Index_Table.java

DBObject_Index_Column.java

DBObject_View.java

DBObject_View_Definition.java

DBObject_Sequence.java

DBObject_Sequence_Counter.java

DBObject_Function.java

DBObject_Function_Argument.java

DBObject_Function_Body.java

DBObject_Operator.java

DBObject_Operator_Signature.java

DBObject_Operator_Definition.java

DBObject_Trigger.java

DBObject_Trigger_Table.java

DBObject_Trigger_Definition.java

Application Dictionary Objects

ADObject_TreeNode.java

Adding Languages and Locales

All messages are contained in the resource file Messages.java, which contains US-English text as default locale.

To add additional languages or locales, copy Messages.java to a new file following Java's Resource Bundle naming convention.

For example, to create a French resource file, name it Messages_fr.java.

To differentiate between French as spoken in France and French as spoken in Canada, create two resource files named Messages_fr_FR.java and Messages_fr_CA.java.

Of course the class declaration must be changed to match the file name, for example public class Messages extends ListResourceBundle { … would become public class Messages_fr_FR extends ListResourceBundle { ….

The file contains an array of {“key”, “localized String”} pairs. The keys should not be modified, as they are used to look up the localized string by the Resource Bundle. The localized string should be translated to the required language.

Note that while Resource Bundles generally accept {“key”, Object} pairs, Migrate can only handle String values such as in {“key”, “String”} pairs⁹.

Adding Database Vendors

To be able to communicate with different database vendors and follow their conventions and rules, Migrate uses a layer of “database engines” which answer to specific predefined requests and provide vendor-specific SQL statements.

These database engines are implemented as Java Interfaces and can therefore easily be extended to other database vendors. In this case, “easily” just means that interfaces for additional database vendors can easily be added, but the actual programming and debugging of such interfaces will still be a laborious task.

The interface definition, manifested in source file DBEngineInterface.java, defines which functions a vendor-specific database engine must contain, what arguments those functions will be given, and what Migrate expects as return values. Consult the [../apidoc/com/kkalice/adempiere/migrate/DBEngineInterface.html DBEngineInterface API] for details (it is generated by javadoc during compilation).

Two database engines are included with the original distribution of Migrate: one for Oracle and one for postgreSQL.

To add a new database engine, it is probably easiest to make a copy of the file which most closely matches the vendor you want to implement, name it according to the new vendor (for example, DBEngine_MySql.java, or DBEngine_AdabasD.java), and rename the class declaration inside the file (public class DBEngine_MySql implements DBEngine_Interface {…, or public class DBEngine_AdabasD implements DBEngine_Interface {…).

Then go through the methods step by step, compare the difference between DBEngine_Oracle.java and DBEngine_Postgresql.java, and figure out what your database vendor requires. After you are done programming the interface, extensive testing and debugging will follow.

To Do

The following are some features which would be nice for Migrate to have, but which have not been implemented yet.

The community is invited to submit contributions:

Identify Renamed Tables

In: Migrate.synchronizeTables()

Migrate drops tables not existing in the reference database and adds tables not existing in the target. So if a table has been renamed, the data contained in that table will be lost. It is therefore necessary to identify tables which have been renamed.

The obvious solution would be to check the AD_Element_ID of the table's primary key, but that method will fail:

In the past, when C_Allocation was renamed to C_AllocationLine, the primary key C_Allocation_ID (element 1380) became C_AllocationHdr_ID, and a new primary key C_AllocationLine_ID (element 2534) was created for the renamed table.

A different solution must be found.

Preserve Parent Links

In: Migrate.preserveParentLinks()

If a table in the live database does not contain a column existing in the reference database, that column will be created with a default value. But if the new column is used as part of a foreign key constraint in the reference database, the default value will not reference any parent record in the target database, which will result in an error when the foreign key is created.

Such "unlinked" fields should be linked to the correct parent, and it must be deduced from other data in the table what the correct parent is.

Currently the hints how to find the correct parent are hard-coded.

At some time, a C_Dunning_ID column was added to the C_DunningRun table, which was used as a foreign key to C_Dunning. When running an upgrade migration, the column is added and filled with 0 as default value. But 0 does not point to any parent in the C_Dunning table, and would thus result in an error when the foreign key is created.

It turns out that C_DunningRun contains a column called C_DunningLevel_ID, which links to the table C_DunningLevel. And C_DunningLevel has a link to the C_Dunning Table. So the correct target for the new C_Dunning_ID column can be deduced by following the link to C_DunningLevel_ID and from there to C_Dunning.

This hint is currently hard-coded.

Migrate should be able to find out by itself how to deduce the correct parent.

As long as that can not be done, such hints must continue to be hard-coded as additional situations of this type are encountered.

Populate New Parents

In: Migrate.populateNewParents()

If new tables exist in the reference database but not in the target, they might be parent tables which must be filled with data from already existing child records.

Originally there was only a table C_Allocation. At some point, that table was renamed C_AllocationLine, and a new parent table C_AllocationHdr was introduced.

At that time, C_AllocationHdr_ID had to be set to the value of C_AllocationLine_ID, and columns in C_AllocationHdr that also existed in C_AllocationLine had to be filled with the values from C_AllocationLine, using

INSERT INTO … SELECT …;

The link from the child to the new parent record had to be set, and since the parent record's C_AllocationHdr_ID now had the same value as the child's C_AllocationLine_ID, it could easily be done with:

UPDATE C_AllocationLine SET C_AllocationHdr_ID = C_AllocationLine_ID WHERE C_AllocationHdr_ID IS NULL;

Finally, any references from other tables pointing to the old child table had to be re-directed to point to the new parent table, for example

UPDATE Fact_Acct SET AD_Table_ID=735 WHERE AD_Table_ID=390;

(C_AllocationHdr has AD_Table_ID 735, C_AllocationLine has AD_Table_ID 390)

Above is actually not so difficult to implement, but the problem is how to find the primary child table.

For example, if C_InvoiceLine and C_InvoiceTax exist, and a new table C_Invoice is created, how do we know that C_InvoiceLine is the table from which C_Invoice should be populated, not C_InvoiceTax?

Another problem arises from inconsistent table naming:

C_Invoice - C_InvoiceLine (the short name is the parent, the long name is the child)

C_AllocationHdr - C_AllocationLine (both parent and child names are long)

GL_JournalBatch - GL_Journal - GL_JournalLine (the parent has a long name, the child has a short name, and the grandchild has a long name again)

Translation of Functions

In: DBEngine_vendor.translateFunctionBodyFull()

Migrate can more or less successfully translate views using regular expressions, but the translation of functions is much more difficult.

Any help to translate functions between the different procedural languages native to each database vendor would be highly appreciated.

Fail-Safe / Safe-Fail

Migrate requires the migration process not to be interrupted.

If it does get interrupted, for example because of a power outage, you need to restore the live database from your backup and start the migration process again from scratch. That is because Migrate drops views, functions, constraints, indexes etc. before starting the migration process. If the migration process is interrupted before those objects are recreated, they will be lost forever.

It would be nice if Migrate saved the meta-data it gathered and then used that saved meta-data to resume migrations which were interrupted.

Delete Client / Delete Transactions

The original Compiere migration tool had a facility to delete transactions (in effect “resetting” a client) or to delete a client entirely. It is probably better not to include such functionality in Migrate but rather have a specialized tool for such kind of task.

However, if anybody sees the need to add such functionality to Migrate, there already is a private dropClient() function in the main Migrate class which can be made public and used for such purpose. (It is currently used to drop the GardenWorld client).

There is no function yet to delete only transactions.

^[8]There is actually only one reason for this limitation: Some JDBC drivers do not return a readable SQL statement but only an object reference when the toString() method is called on a prepared statement, rendering it useless for logging purposes. Therefore Migrate uses a wrapper around the PreparedStatement class, which overrides the toString() method and returns a human readable string to be used for logging. All other methods are caught to extract variable information which is used to generate the string, and then passed on to the original PreparedStatement class. As Java evolves, new methods are added to PreparedStatement, some of which also accept or return classes introduced in the new Java version. Since Java does not allow conditional compiling, a choice had to be made whether to be compatible with previous versions or with the newest version, and the decision went in favor of the newest version.

^[9]For this reason, to translate keyboard codes for mnemonic highlighting of menu items, labels, or buttons, the keyboard code, which is an int, is converted to an Integer which is converted to a String, as in:

	…
	{"guiMenuHelp", "Help"},
	{"guiMenuHelpMnemonic", new Integer(KeyEvent.VK_H).toString()},
	…

Prev
Chapter 3. Migrating a Database	Home

@@ Line 3: / Line 3: @@
 <div>
-=<span id="N10C6B">Compiling and Extending</span>=
+=<span id="N10C29">Compiling and Extending</span>=
 </div><div><div class="titlepage"><div><div>
-==<span id="N10C6F">Compiling Migrate</span>==
+==<span id="N10C2D">Compiling Migrate</span>==
 </div></div></div><p>Normally there should be no need to compile <span class="productname">Migrate</span>, as it will be installed together with <span class="productname">Adempiere</span>.</p><p>However, there may be situations when you separately want to compile <span class="productname">Migrate</span>, either to modify the code to suit your personal needs, or to fix bugs or extend the code and hopefully contribute your enhancements to the <span class="productname">Adempiere</span> project.</p><div><div class="titlepage"><div><div>
+===<span id="N10C44">Requirements</span>===
-:''It seems there are ADempiere versions without this source. Regarding to [[https://sourceforge.net/projects/adempiere/forums/forum/610547/topic/5274034 this forum thread]] you can download the migrate tool from http://adempiere.hg.sourceforge.net/hgweb/adempiere/contribution_kkalice_migrate/ ([[User:Tbayen]])''
+</div></div></div><p><span class="productname">Migrate</span> requires the <span class="productname">Java Development Kit</span> version 1.7 (JDK 7)<sup>[[#ftn.N10C4F|8]]</sup> and therefore also at least version 3.7.0lts of <span class="productname">Adempiere</span>.</p></div><div><div class="titlepage"><div><div>
+===<span id="N10C65">Downloading and Compiling the Source Code</span>===
-===<span id="N10C86">Requirements</span>===
+</div></div></div><div><ol class="procedure" type="1"><li><p>Download the <span class="productname">Adempiere</span> source.</p><div class="informalexample"><p><span class="command"><strong>hg clone http://adempiere.hg.sourceforge.net/hgroot/adempiere/adempiere#development .</strong></span></p></div></li><li><p>You can either compile the complete <span class="productname">Adempiere</span> project or only the <span class="productname">Migrate</span> sub-project.</p><ul>
-</div></div></div><p><span class="productname">Migrate</span> requires the <span class="productname">Java Development Kit</span> version 1.6 (JDK 6)<sup>[[#ftn.N10C91|10]]</sup> and therefore also at least version 3.5.3a of <span class="productname">Adempiere</span>.</p></div><div><div class="titlepage"><div><div>
-===<span id="N10CA7">Compiling within Adempiere Source Code</span>===
-</div></div></div><div><ol class="procedure" type="1"><li><p>Download the <span class="productname">Adempiere</span> source.</p><div class="informalexample"><p><span class="command"><strong>svn export http://adempiere.svn.sourceforge.net/svnroot/adempiere/trunk</strong></span></p></div></li><li><p>You can either compile the complete <span class="productname">Adempiere</span> project or only the <span class="productname">Migrate</span> sub-project.</p><ul>
 							<li><p>To compile the complete <span class="productname">Adempiere</span> project, change to directory <code class="filename">utils_dev</code>.</p><div class="informalexample"><p><span class="command"><strong>cd utils_dev</strong></span></p></div></li>
 							<li><p>To compile only the <span class="productname">Migrate</span> sub-project, change to directory <code class="filename">migrate</code>.</p><div class="informalexample"><p><span class="command"><strong>cd migrate</strong></span></p></div></li>
 						</ul></li><li><p>Then execute <span class="command"><strong>RUN_build.sh</strong></span> (or <span class="command"><strong>RUN_build.bat</strong></span>).</p><div class="informalexample"><p><span class="command"><strong>./RUN_build.sh</strong></span></p></div></li><li><p>The resulting JAR file (<code class="filename">migrate.jar</code>) will be created in the <code class="filename">migrate</code> project directory and also copied to the <code class="filename">../lib</code> directory.</p></li><li><p>This will also generate the API and user documentation, to be found in the <code class="filename">migrate/apidoc</code> and <code class="filename">migrate/userdoc</code> directories, respectively.</p></li></ol></div><p>For details on how to work with <span class="productname">Adempiere</span> source code, consult the <span class="productname">Adempiere</span>[http://www.adempiere.com/index.php/Compile documentation].</p></div><div><div class="titlepage"><div><div>
-===<span id="N10D0D">Compiling as Stand-Alone Application</span>===
+===<span id="N10CC8">Building and Running Migrate in Eclipse</span>===
-</div></div></div><div><ol class="procedure" type="1"><li><p>Download only the <span class="productname">Migrate</span> sub-project source code.</p><div class="informalexample"><p><span class="command"><strong>svn export http://adempiere.svn.sourceforge.net/svnroot/adempiere/trunk/migrate</strong></span></p></div></li><li><p>Change to the <code class="filename">migrate</code> directory.</p><div class="informalexample"><p><span class="command"><strong>cd migrate</strong></span></p></div></li><li><p>Then execute <span class="command"><strong>RUN_build.sh</strong></span> (or <span class="command"><strong>RUN_build.bat</strong></span>).</p><div class="informalexample"><p><span class="command"><strong>./RUN_build.sh</strong></span></p></div></li><li><p>The resulting JAR file (<code class="filename">migrate.jar</code>) will be in the current directory.</p></li><li><p>This will also generate the API and user documentation, to be found in the <code class="filename">apidoc</code> and <code class="filename">userdoc</code> subdirectories, respectively.</p></li></ol></div></div><div><div class="titlepage"><div><div>
-===<span id="N10D42">Building and Running Migrate in Eclipse</span>===
 </div></div></div><p>Consult the <span class="productname">Adempiere</span>[http://www.adempiere.com/index.php/Create_your_ADempiere_development_environment documentation] on how to compile and run <span class="productname">Adempiere</span> from within <span class="productname">ECLIPSE</span>.</p><p>Note that the <span class="productname">JDBC</span> drivers for installed databases must be in the classpath.</p><p>If you have installed <span class="productname">Adempiere</span>, they can be found in <code class="filename">$ADEMPIERE_HOME/lib</code>:
 					</p><div><ul class="itemizedlist" type="disc" compact><li><p><code class="filename">$ADEMPIERE_HOME/lib/oracle.jar</code> for <span class="productname">Oracle</span>
@@ Line 29: / Line 24: @@
 							</p></li></ul></div><p>
 				</p><p>To add files or directories to the <code class="envar">classpath</code> in <span class="productname">Eclipse</span> (version 3.4.1), in the <span class="guimenu"><span class="accel">R</span>un</span> menu select <span class="guimenuitem">Ru<span class="accel">n</span>&nbsp;Configurations&hellip;</span>, select the <span class="guilabel">Classpath</span> tab and click the <span class="guibutton">Add&nbsp;E<span class="accel">x</span>ternal&nbsp;JARs&hellip;</span> button.</p><span id="doc_eclipse">[[File:doc_eclipse.png|thumb|none|JDBC drivers must be set in the classpath for Migrate to run in Eclipse]]</span></div></div><div><div class="titlepage"><div><div>
+==<span id="N10D4E">Extending Migrate</span>==
-==<span id="N10DC8">Extending Migrate</span>==
 </div></div></div><div><div class="titlepage"><div><div>
-===<span id="N10DCF">Source Files</span>===
+===<span id="N10D55">Source Files</span>===
 </div></div></div><p>Being open-source, <span class="productname">Migrate</span> has the advantage that you can modify the source code to fit your particular needs.</p><p>More than that, <span class="productname">Migrate</span> is designed to be easily extendable for localization and for handling additional database vendors, and you are invited to help and contribute your solutions to <span class="productname">Adempiere</span>.</p><p>To help you navigate the source files, they are listed here by category:</p><div class="table"><span id="tblSourceFiles"></span><p class="title"><b>Table&nbsp;4.1.&nbsp;Source Files</b></p><div class="table-contents"><table summary="Source Files" border="1"><tr><th align="center">Category</th><th align="center">Source Files</th></tr><tr><td align="left">
 									Main class
@@ Line 182: / Line 176: @@
 									</p>
 								</td></tr></table></div></div><br class="table-break"></div><div><div class="titlepage"><div><div>
-===<span id="N10F8A">Adding Languages and Locales</span>===
+===<span id="N10F10">Adding Languages and Locales</span>===
 </div></div></div><p>All messages are contained in the resource file <code class="filename">Messages.java</code>, which contains US-English text as default locale.</p><p>To add additional languages or locales, copy <code class="filename">Messages.java</code> to a new file following <span class="productname">Java</span>'s [http://java.sun.com/developer/technicalArticles/Intl/ResourceBundles/ Resource Bundle] naming convention.</p><p>For example, to create a French resource file, name it <code class="filename">Messages_fr.java</code>.</p><p>To differentiate between French as spoken in France and French as spoken in Canada, create two resource files named <code class="filename">Messages_fr_FR.java</code> and <code class="filename">Messages_fr_CA.java</code>.</p><p>Of course the class declaration must be changed to match the file name, for example <code class="code">public class <em>Messages</em> extends ListResourceBundle { &hellip;</code> would become <code class="code">public class <em>Messages_fr_FR</em> extends ListResourceBundle { &hellip;</code>.
-				</p><p>The file contains an array of <code class="code">{&ldquo;<em>key</em>&rdquo;, &ldquo;<em>localized String</em>&rdquo;}</code> pairs. The keys should not be modified, as they are used to look up the localized string by the Resource Bundle. The localized string should be translated to the required language.</p><p>Note that while Resource Bundles generally accept <code class="code">{&ldquo;<em>key</em>&rdquo;, <em>Object</em>}</code> pairs, <span class="productname">Migrate</span> can only handle String values such as in <code class="code">{&ldquo;<em>key</em>&rdquo;, &ldquo;<em>String</em>&rdquo;}</code> pairs<sup>[[#ftn.N10FDC|11]]</sup>.</p></div><div><div class="titlepage"><div><div>
+				</p><p>The file contains an array of <code class="code">{&ldquo;<em>key</em>&rdquo;, &ldquo;<em>localized String</em>&rdquo;}</code> pairs. The keys should not be modified, as they are used to look up the localized string by the Resource Bundle. The localized string should be translated to the required language.</p><p>Note that while Resource Bundles generally accept <code class="code">{&ldquo;<em>key</em>&rdquo;, <em>Object</em>}</code> pairs, <span class="productname">Migrate</span> can only handle String values such as in <code class="code">{&ldquo;<em>key</em>&rdquo;, &ldquo;<em>String</em>&rdquo;}</code> pairs<sup>[[#ftn.N10F62|9]]</sup>.</p></div><div><div class="titlepage"><div><div>
-===<span id="N10FE3">Adding Database Vendors</span>===
+===<span id="N10F69">Adding Database Vendors</span>===
 </div></div></div><p>To be able to communicate with different database vendors and follow their conventions and rules, <span class="productname">Migrate</span> uses a layer of &ldquo;database engines&rdquo; which answer to specific predefined requests and provide vendor-specific SQL statements.</p><p>These database engines are implemented as <span class="productname">Java</span> Interfaces and can therefore easily be extended to other database vendors. In this case, &ldquo;easily&rdquo; just means that interfaces for additional database vendors can easily be added, but the actual programming and debugging of such interfaces will still be a laborious task.</p><p>The interface definition, manifested in source file <code class="filename">DBEngineInterface.java</code>, defines which functions a vendor-specific database engine must contain, what arguments those functions will be given, and what <span class="productname">Migrate</span> expects as return values. Consult the [../apidoc/com/kkalice/adempiere/migrate/DBEngineInterface.html DBEngineInterface API] for details (it is generated by javadoc
 during compilation).</p><p>Two database engines are included with the original distribution of <span class="productname">Migrate</span>: one for <span class="productname">Oracle</span> and one for <span class="productname">postgreSQL</span>.</p><p>To add a new database engine, it is probably easiest to make a copy of the file which most closely matches the vendor you want to implement, name it according to the new vendor (for example, <code class="filename">DBEngine_<em>MySql</em>.java</code>, or <code class="filename">DBEngine_<em>AdabasD</em>.java</code>), and rename the class declaration inside the file (<code class="code">public class DBEngine_<em>MySql</em> implements DBEngine_Interface {&hellip;</code>, or <code class="code">public class DBEngine_<em>AdabasD</em> implements DBEngine_Interface {&hellip;</code>).</p><p>Then go through the methods step by step, compare the difference between <code class="filename">DBEngine_Oracle.java</code> and <code class="filename">DBEngine_Postgresql.java</code>, and figure out what your database vendor requires. After you are done programming the interface, extensive testing and debugging will follow.</p></div><div><div class="titlepage"><div><div>
-===<span id="N1102A">To Do</span>===
+===<span id="N10FB0">To Do</span>===
 </div></div></div><p>The following are some features which would be nice for <span class="productname">Migrate</span> to have, but which have not been implemented yet.</p><p>The community is invited to submit contributions:</p><div><div class="titlepage"><div><div>
-====<span id="N11035">Identify Renamed Tables</span>====
+====<span id="N10FBB">Identify Renamed Tables</span>====
 </div></div></div><p>In: <code class="classname">Migrate</code>.<code class="function">synchronizeTables()</code></p><p><span class="productname">Migrate</span> drops tables not existing in the reference database and adds tables not existing in the target. So if a table has been renamed, the data contained in that table will be lost. It is therefore necessary to identify tables which have been renamed.</p><p>The obvious solution would be to check the <code class="varname">AD_Element_ID</code> of the table's primary key, but that method will fail:</p><div class="informalexample"><p>In the past, when <code class="filename">C_Allocation</code> was renamed to <code class="filename">C_AllocationLine</code>, the primary key <code class="varname">C_Allocation_ID</code> (element 1380) became <code class="varname">C_AllocationHdr_ID</code>, and a new primary key <code class="varname">C_AllocationLine_ID</code> (element 2534) was created for the renamed table.</p></div><p>A different solution must be found.</p></div><div><div class="titlepage"><div><div>
-====<span id="N1105D">Preserve Parent Links</span>====
+====<span id="N10FE3">Preserve Parent Links</span>====
 </div></div></div><p>In: <code class="classname">Migrate</code>.<code class="function">preserveParentLinks()</code></p><p>If a table in the live database does not contain a column existing in the reference database, that column will be created with a default value. But if the new column is used as part of a foreign key constraint in the reference database, the default value will not reference any parent record in the target database, which will result in an error when the foreign key is created.</p><p>Such "unlinked" fields should be linked to the correct parent, and it must be deduced from other data in the table what the correct parent is.</p><p>Currently the hints how to find the correct parent are hard-coded.</p><div class="informalexample"><p>At some time, a <code class="varname">C_Dunning_ID</code> column was added to the <code class="filename">C_DunningRun</code> table, which was used as a foreign key to <code class="filename">C_Dunning</code>. When running an upgrade migration, the column is added and filled with <code class="constant">0</code> as default value. But <code class="constant">0</code> does not point to any parent in the <code class="filename">C_Dunning</code> table, and would thus result in an error when the foreign key is created.</p><p>It turns out that <code class="filename">C_DunningRun</code> contains a column called <code class="varname">C_DunningLevel_ID</code>, which links to the table <code class="filename">C_DunningLevel</code>. And <code class="filename">C_DunningLevel</code> has a link to the <code class="filename">C_Dunning</code> Table. So the correct target for the new <code class="varname">C_Dunning_ID</code> column can be deduced by following the link to <code class="varname">C_DunningLevel_ID</code> and from there to <code class="filename">C_Dunning</code>.</p><p>This hint is currently hard-coded.</p></div><p><span class="productname">Migrate</span> should be able to find out by itself how to deduce the correct parent.</p><p>As long as that can not be done, such hints must continue to be hard-coded as additional situations of this type are encountered.</p></div><div><div class="titlepage"><div><div>
-====<span id="N110A5">Populate New Parents</span>====
+====<span id="N1102B">Populate New Parents</span>====
 </div></div></div><p>In: <code class="classname">Migrate</code>.<code class="function">populateNewParents()</code></p><p>If new tables exist in the reference database but not in the target, they might be parent tables which must be filled with data from already existing child records.</p><div class="informalexample"><p>Originally there was only a table <code class="filename">C_Allocation</code>. At some point, that table was renamed <code class="filename">C_AllocationLine</code>, and a new parent table <code class="filename">C_AllocationHdr</code> was introduced.</p><p>At that time, <code class="varname">C_AllocationHdr_ID</code> had to be set to the value of <code class="varname">C_AllocationLine_ID</code>, and columns in <code class="filename">C_AllocationHdr</code> that also existed in <code class="filename">C_AllocationLine</code> had to be filled with the values from <code class="filename">C_AllocationLine</code>, using</p><p>
 							<code class="code">INSERT INTO &hellip; SELECT &hellip;;</code>
@@ Line 204: / Line 198: @@
 						</p><table summary="Simple list" border="0"><tr><td><code class="filename">C_Invoice</code> - <code class="filename">C_InvoiceLine</code> (the short name is the parent, the long name is the child)</td></tr><tr><td><code class="filename">C_AllocationHdr</code> - <code class="filename">C_AllocationLine</code> (both parent and child names are long)</td></tr><tr><td><code class="filename">GL_JournalBatch</code> - <code class="filename">GL_Journal</code> - <code class="filename">GL_JournalLine</code> (the parent has a long name, the child has a short name, and the grandchild has a long name again)</td></tr></table><p>
 					</p></div><div><div class="titlepage"><div><div>
-====<span id="N1112F">Translation of Functions</span>====
+====<span id="N110B5">Translation of Functions</span>====
 </div></div></div><p>In: <code class="classname">DBEngine_<em>vendor</em></code>.<code class="function">translateFunctionBodyFull()</code></p><p><span class="productname">Migrate</span> can more or less successfully translate views using regular expressions, but the translation of functions is much more difficult.</p><p>Any help to translate functions between the different procedural languages native to each database vendor would be highly appreciated.</p></div><div><div class="titlepage"><div><div>
-====<span id="N11142">Fail-Safe / Safe-Fail</span>====
+====<span id="N110C8">Fail-Safe / Safe-Fail</span>====
 </div></div></div><p><span class="productname">Migrate</span> requires the migration process not to be interrupted.</p><p>If it does get interrupted, for example because of a power outage, you need to restore the live database from your backup and start the migration process again from scratch. That is because <span class="productname">Migrate</span> drops views, functions, constraints, indexes etc. before starting the migration process. If the migration process is interrupted before those objects are recreated, they will be lost forever.</p><p>It would be nice if <span class="productname">Migrate</span> saved the meta-data it gathered and then used that saved meta-data to resume migrations which were interrupted.</p></div><div><div class="titlepage"><div><div>
-====<span id="N11154">Delete Client / Delete Transactions</span>====
+====<span id="N110DA">Delete Client / Delete Transactions</span>====
-</div></div></div><p>The original <span class="productname">Compiere</span> migration tool had a facility to delete transactions (in effect &ldquo;resetting&rdquo; a client) or to delete a client entirely. It is probably better not to include such functionality in <span class="productname">Migrate</span> but rather have a specialized tool for such kind of task.</p><p>However, if anybody sees the need to add such functionality to <span class="productname">Migrate</span>, there already is a private <code class="function">dropClient()</code> function in the main <code class="classname">Migrate</code> class which can be made public and used for such purpose. (It is currently used to drop the <span class="emphasis"><em>GardenWorld</em></span> client).</p><p>There is no function yet to delete only transactions.</p></div></div></div><div class="footnotes"><br><hr align="left" width="100"><div><p><sup>[<span id="ftn.N10C91" class="para">10</span>] </sup>There is actually only one reason for this limitation: Some <span class="productname">JDBC</span> drivers do not return a readable SQL statement but only an object reference when the toString() method is called on a prepared statement, rendering it useless for logging purposes. Therefore <span class="productname">Migrate</span> uses a wrapper around the PreparedStatement class, which overrides the toString() method and returns a human readable string to be used for logging. All other methods are caught to extract variable information which is used to generate the string, and then passed on to the original PreparedStatement class. In <span class="productname">Java</span> 1.6, some new methods were added to PreparedStatement, which also accept or return classes new to <span class="productname">Java</span> 1.6. Since <span class="productname">Java</span> does not allow conditional compiling, a choice had to be made whether to be compatible with version 1.5 or version 1.6. Naturally, a choice was made for the newer version.</p></div><div><p><sup>[<span id="ftn.N10FDC" class="para">11</span>] </sup>For this reason, to translate keyboard codes for mnemonic highlighting of menu items, labels, or buttons, the keyboard code, which is an int, is converted to an Integer which is converted to a String, as in:
+</div></div></div><p>The original <span class="productname">Compiere</span> migration tool had a facility to delete transactions (in effect &ldquo;resetting&rdquo; a client) or to delete a client entirely. It is probably better not to include such functionality in <span class="productname">Migrate</span> but rather have a specialized tool for such kind of task.</p><p>However, if anybody sees the need to add such functionality to <span class="productname">Migrate</span>, there already is a private <code class="function">dropClient()</code> function in the main <code class="classname">Migrate</code> class which can be made public and used for such purpose. (It is currently used to drop the <span class="emphasis"><em>GardenWorld</em></span> client).</p><p>There is no function yet to delete only transactions.</p></div></div></div><div class="footnotes"><br><hr align="left" width="100"><div><p><sup>[<span id="ftn.N10C4F" class="para">8</span>] </sup>There is actually only one reason for this limitation: Some <span class="productname">JDBC</span> drivers do not return a readable SQL statement but only an object reference when the toString() method is called on a prepared statement, rendering it useless for logging purposes. Therefore <span class="productname">Migrate</span> uses a wrapper around the PreparedStatement class, which overrides the toString() method and returns a human readable string to be used for logging. All other methods are caught to extract variable information which is used to generate the string, and then passed on to the original PreparedStatement class. As <span class="productname">Java</span> evolves, new methods are added to PreparedStatement, some of which also accept or return classes introduced in the new <span class="productname">Java</span> version. Since <span class="productname">Java</span> does not allow conditional compiling, a choice had to be made whether to be compatible with previous versions or with the newest version, and the decision went in favor of the newest version.</p></div><div><p><sup>[<span id="ftn.N10F62" class="para">9</span>] </sup>For this reason, to translate keyboard codes for mnemonic highlighting of menu items, labels, or buttons, the keyboard code, which is an int, is converted to an Integer which is converted to a String, as in:
-<pre id="N10FDF">	&hellip;
+<pre id="N10F65">	&hellip;
 	{"guiMenuHelp", "Help"},
 	{"guiMenuHelpMnemonic", new Integer(KeyEvent.VK_H).toString()},

Difference between revisions of "Migrate - Compiling and Extending"

Latest revision as of 06:41, 12 November 2012

Contents

Compiling and Extending

Compiling Migrate

Requirements

Downloading and Compiling the Source Code

Building and Running Migrate in Eclipse

Extending Migrate

Source Files

Adding Languages and Locales

Adding Database Vendors

To Do

Identify Renamed Tables

Preserve Parent Links

Populate New Parents

Translation of Functions

Fail-Safe / Safe-Fail

Delete Client / Delete Transactions

Navigation menu

Personal tools

Namespaces

Variants

Views

More

Search

Navigation

Tools