Difference between revisions of "FR3409987 Cleanup of Ant build scripts"
(→Introduction) |
m (moved Feature: Cleanup of Ant build scripts to FR3409987 Cleanup of Ant build scripts: Aligning name with other FR pages) |
||
(3 intermediate revisions by 2 users not shown) | |||
Line 8: | Line 8: | ||
==Modified Ant build scripts== | ==Modified Ant build scripts== | ||
− | ADempiere | + | ADempiere 370LTS contains the following Ant build scripts which are required to build the entire system: |
**tools/build.xml | **tools/build.xml | ||
Line 106: | Line 106: | ||
The goal of that first cleanup was to keep the ADempiere build output structure as similar to the already existing structure as possible. Nonetheless there are some differences which are listed in the following table. The table lists all orphans, i.e. files that are only available on one side. | The goal of that first cleanup was to keep the ADempiere build output structure as similar to the already existing structure as possible. Nonetheless there are some differences which are listed in the following table. The table lists all orphans, i.e. files that are only available on one side. | ||
− | This list has been created by a compare tool (“Beyond Compare”) and it includes the content of all jar and war files. The left side lists all files of the ADempiere | + | This list has been created by a compare tool (“Beyond Compare”) and it includes the content of all jar and war files. The left side lists all files of the ADempiere 370LTS release that are not available in a build created using the cleaned up Ant build scripts and vice versa. |
{|cellpadding="5" cellspacing="0" border="1" | {|cellpadding="5" cellspacing="0" border="1" | ||
|- | |- | ||
− | ! Released ADempiere | + | ! Released ADempiere 370LTS |
− | ! ADempiere | + | ! ADempiere 370LTS created using the modified scripts |
|- valign="top" | |- valign="top" | ||
| | | | ||
Line 120: | Line 120: | ||
INDEX.LIST | INDEX.LIST | ||
org | org | ||
+ | adempiere | ||
+ | process | ||
+ | DBA_PrepareRelease.class | ||
compiere | compiere | ||
interfaces | interfaces | ||
Line 137: | Line 140: | ||
SendByEmailListener.class | SendByEmailListener.class | ||
SwingJRViewerProvider.class | SwingJRViewerProvider.class | ||
+ | swing | ||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
utils | utils | ||
DigestOfFile.class | DigestOfFile.class | ||
| | | | ||
+ | lib | ||
+ | Adempiere.jar | ||
+ | META-INF | ||
+ | |||
+ | org | ||
+ | adempiere | ||
+ | process | ||
+ | |||
+ | compiere | ||
+ | interfaces | ||
+ | |||
+ | report | ||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | swing | ||
+ | CComboBox$1.class | ||
+ | CComboBox$2.class | ||
+ | CComboBox$3.class | ||
+ | CComboBox$ReducibleKeyListener$1.class | ||
+ | CComboBox$ReducibleKeyListener$2.class | ||
+ | CComboBox$ReducibleKeyListener.class | ||
+ | CComboBox$ReducibleModel.class | ||
+ | |||
+ | |||
|- valign="top" | |- valign="top" | ||
| | | | ||
Line 146: | Line 192: | ||
META-INF | META-INF | ||
INDEX.LIST | INDEX.LIST | ||
− | |||
| | | | ||
+ | lib | ||
+ | CInstall | ||
+ | META-INF | ||
+ | |||
|- valign="top" | |- valign="top" | ||
| | | | ||
Line 186: | Line 235: | ||
lib | lib | ||
posterita.jar | posterita.jar | ||
+ | META-INF | ||
+ | INDEX.LIST | ||
| | | | ||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
|- valign="top" | |- valign="top" | ||
| | | | ||
lib | lib | ||
posteritaPatches.jar | posteritaPatches.jar | ||
+ | META-INF | ||
+ | MANIFEST.MF | ||
| | | | ||
|- valign="top" | |- valign="top" | ||
Line 218: | Line 252: | ||
RUN_build.bat | RUN_build.bat | ||
| | | | ||
+ | lib | ||
+ | webui.war | ||
+ | |||
+ | |||
|} | |} | ||
− | Note that the “Adempiere.jar” on the left side contains classes not available on the right side. These classes are contained in “CompiereJasper.jar” which is only available on the right side. Note also that there is no “INDEX.LIST” available because “Adempiere.jar” is not indexed. | + | Note that the “Adempiere.jar” on the left side contains classes not available on the right side. These classes are contained in “CompiereJasper.jar” which is only available on the right side. Note also that there is no “INDEX.LIST” available because “Adempiere.jar” is not indexed. Indexing jar files at this level is not done anymore because the files will be repacked anyway once the ADempiere setup is started. |
− | There are some files included on the left side that simply should not be included, namely | + | There are some files included on the left side that simply should not be included, namely “build.xml” and “RUN_build.bat” which are part of “webui.war”. These files are not included anymore if a build is created using the cleaned up Ant build scripts. |
“posterita.jar” and “posteritaPatches.jar” are not available at all on the right side. These files are not created anymore since the compiled classes of posterita are directly packaged into “posterita.war”. | “posterita.jar” and “posteritaPatches.jar” are not available at all on the right side. These files are not created anymore since the compiled classes of posterita are directly packaged into “posterita.war”. | ||
− | |||
==Dependency Graph== | ==Dependency Graph== | ||
Using the classpaths of the cleaned up Ant build scripts it is possible to derive a [[:File:ADempiere_dependency_graph.png|dependency graph]] of the ADempiere jar and war files. | Using the classpaths of the cleaned up Ant build scripts it is possible to derive a [[:File:ADempiere_dependency_graph.png|dependency graph]] of the ADempiere jar and war files. | ||
− | [[File:ADempiere_dependency_graph.png | + | [[File:ADempiere_dependency_graph.png]] |
As can be seen on this graph there are some redundancies. “CSTools” (probably short for “Compiere Server Tools”) and “CCTools” (probably short for “Compiere Client Tools”) contain almost the same libraries. | As can be seen on this graph there are some redundancies. “CSTools” (probably short for “Compiere Server Tools”) and “CCTools” (probably short for “Compiere Client Tools”) contain almost the same libraries. | ||
Line 237: | Line 274: | ||
==ADempiere setup== | ==ADempiere setup== | ||
===Current situation=== | ===Current situation=== | ||
− | In ADempiere | + | In ADempiere 370LTS the build and setup processes are tightly coupled. It all starts with the invocation of “utils_dev/build.xml”. This master build script will invoke the build scripts of the individual ADempiere subprojects which will create the jar and war files. The final step done as part of this [[:File:ADempiere_build.png|process]] is the creation of the build output folder in “<ADempiereRoot>/install”. |
In order to install ADempiere this output folder is copied to the target machine that will actually serve as host for ADempiere. On this machine either “RUN_setup[.bat|.sh]” or “RUN_silentsetup[.bat|.sh]” will be executed. This in turn will call the class “org.compiere.install.Setup” or “org.compiere.install.SilentSetup”. | In order to install ADempiere this output folder is copied to the target machine that will actually serve as host for ADempiere. On this machine either “RUN_setup[.bat|.sh]” or “RUN_silentsetup[.bat|.sh]” will be executed. This in turn will call the class “org.compiere.install.Setup” or “org.compiere.install.SilentSetup”. | ||
− | Both of these classes will then invoke Ant with “build.xml” in the ADempiere output folder. This file has originally been copied from “<ADempiereRoot>/install/Adempiere” and it will repack the jar and war files into new jar and war files as shown [[:File:ADempiere_output_structure_2.png | + | Both of these classes will then invoke Ant with “build.xml” in the ADempiere output folder. This file has originally been copied from “<ADempiereRoot>/install/Adempiere” and it will repack the jar and war files into new jar and war files as shown [[:File:ADempiere_output_structure_2.png]]. Note that any customizations for ADempiere will be included as part of this step. Note that this step is actually a build step. |
In addition to repacking the jar and war files on the host machine “build.xml” will create a bunch of xml and batch / shell files out of templates. These templates contain variables which are replaced with values (depending on the target environment, e.g. ports) by “build.xml”. | In addition to repacking the jar and war files on the host machine “build.xml” will create a bunch of xml and batch / shell files out of templates. These templates contain variables which are replaced with values (depending on the target environment, e.g. ports) by “build.xml”. | ||
Line 277: | Line 314: | ||
[[File:ADempiere_build.png|800px]] | [[File:ADempiere_build.png|800px]] | ||
− | [[File:ADempiere_output_structure_2.png | + | [[File:ADempiere_output_structure_2.png]] |
===Proposed setup changes=== | ===Proposed setup changes=== | ||
Line 288: | Line 325: | ||
==Conclusions== | ==Conclusions== | ||
The cleaned up ADempiere Ant build scripts are ready to be used. As a next step the packaging structure of ADempiere should be redesigned to remove redundancies whenever possible. Once this is done the repackaging steps, which are now done as part of the ADempiere setup, should be done as part of a build, not a setup step. This will pave the way to finally come up with clean, platform dependent installers. | The cleaned up ADempiere Ant build scripts are ready to be used. As a next step the packaging structure of ADempiere should be redesigned to remove redundancies whenever possible. Once this is done the repackaging steps, which are now done as part of the ADempiere setup, should be done as part of a build, not a setup step. This will pave the way to finally come up with clean, platform dependent installers. | ||
+ | |||
+ | [[Category:Feature Request - In Progress]] |
Latest revision as of 09:32, 7 December 2011
Contents
Introduction
The central idea behind this change is to simplify and cleanup the build, installation and deployment processes of ADempiere. As a first step towards this goal the change discussed in this paper modifies the existing Ant build scripts of ADempiere. The paper then discusses additional proposed but not yet implemented changes.
Note that this wiki page is also available as pdf.
Modified Ant build scripts
ADempiere 370LTS contains the following Ant build scripts which are required to build the entire system:
- tools/build.xml
- base/build.xml
- extend/build.xml
- client/build.xml
- JasperReports/build.xml
- serverRoot/build.xml
- serverApps/build.xml
- webStore/build.xml
- webCM/build.xml
- sqlj/build.xml
- posterita/posterita/build.xml
- zkwebui/build.xml
- install/build.xml
These project specific Ant scripts are invoked by the master build script “utils_dev/build.xml”. (Note that the scripts listed above are listed in the order they are called by the master build script)
All of the above listed build files have been modified as part of this proposed change.
Build script modifications
Compiler flags
All Java compiler calls in the Ant build scripts (via javac Ant task) now use the same arguments. To achieve this, the following properties have been added to “utils_dev/build.properties”:
compile.version=1.6 compile.debug=false compile.optimize=true compile.deprecation=true compile.compress=false
"compile.version" is used to instruct the Java compiler to generate code for this version of the Java Virtual Machine (JVM).
"compile.debug" which is now set to false will prevent the compiler from adding debug information to compiled classes. Note that it is set to false since this setting shall be used to create ADempiere releases. During development it might be necessary to include debug information. In such a case this setting should be modified locally.
"compile.optimize" instructs the compiler to optimize the generated code. Note that this switch might also need to be turned off during development but shall be turned on whenever a release of ADempiere is created.
“compile.deprecation” which is set to true will show compile time warnings whenever deprecated code is used.
“compile.compress” has the effect that resulting jar, war and ear files are compressed. This will result in slightly smaller files but might have the effect that loading time increases. Because of this the default setting is “false”.
All “javac” task invocations in the Ant build scripts utilize these additional properties and typically look as follows:
<javac srcdir="${src.dir}" destdir="${build.dir}" target="${compile.version}" debug="${compile.debug}" optimize="${compile.optimize}" deprecation="${compile.deprecation}" includeAntRuntime="false" classpathref="project.classpath"/>
Elimination of circular dependency
The Ant build scripts contained one circular dependency:
Compiling the classes that shall be packed into “CompiereJasper.jar” requires the compiled classes which will be packed into Adempiere.jar, i.e. the compiled classes of the “client” subproject. Adempiere.jar however shall contain both the content of CompiereJasper.jar and the classes of the “client” subproject.
So far the existing approach to build was as follows:
- “utils_dev/build.xml” invokes “client/build.xml” to compile the classes of the “client” subproject
- “client/build.xml” invokes “JasperReports/build.xml” to build the “JasperReports” subproject
- “client/build.xml” builds “Adempiere.jar” since both the compiled classes of the “client” and the “JasperReports” subprojects are now available
- “utils_dev/build.xml” needlessly invokes “JasperReports/build.xml”
Since “Adempiere.jar” will be repackaged by “install/Adempiere/build.xml” anyway there is no need for this complicated procedure.
In the cleaned up Ant build scripts “Adempiere.jar” is created before “CompiereJasper.jar” and therefore does not include the compiled classes of the “JasperReports” subproject.
“install/Adempiere/build.xml” has been modified so that “Adempiere.jar” contains the content of “JasperReports.jar” after “Adempiere.jar” has been repacked.
Exclusion of unnecessary build scripts
In addition to the Ant build scripts listed in Modified Ant build scripts the ADempiere project contained the Ant build file “JasperReportsWebApp/build.xml”. Invocation of this build script in “utils_dev/build.xml” has been removed since it does not create files that are used by ADempiere. None of the files created by “JasperReportsWebApp/build.xml” are referenced anywhere nor are they packed into the ADempiere jar, war or ear files.
Classpath
The classpath for all modified Ant build scripts has been minimized, i.e. all unnecessary entries in the classpaths of the scripts have been removed.
Creation of output structure
The output folder containing the ADempiere build has so far been created by the Ant build script “install/build.xml”. This functionality has been moved to the master build file “utils_dev/build.xml”. This has been done because this seems to be a more logical place to perform this.
Moved files
The Ant build file “posterita/posterita/build.xml” has been moved to “posterita/build.xml”.
Note that the ADempiere output folder is now located in “<ADempiereRoot>/build” instead of “<ADempiereRoot>/install”.
General
In general it has been attempted to make the structure of the build scripts as straightforward as possible. All modified build scripts use the same structure, target names, etc.
Diff between old and new build output
The goal of that first cleanup was to keep the ADempiere build output structure as similar to the already existing structure as possible. Nonetheless there are some differences which are listed in the following table. The table lists all orphans, i.e. files that are only available on one side.
This list has been created by a compare tool (“Beyond Compare”) and it includes the content of all jar and war files. The left side lists all files of the ADempiere 370LTS release that are not available in a build created using the cleaned up Ant build scripts and vice versa.
Released ADempiere 370LTS | ADempiere 370LTS created using the modified scripts |
---|---|
lib Adempiere.jar META-INF INDEX.LIST org adempiere process DBA_PrepareRelease.class compiere interfaces MD5.class report Barbecue.class ExportListener.class JasperReportViewer.class JasperViewer$1.class JasperViewer$2.class JasperViewer.class JRViewerProvider.class ReportStarter$FileFilter.class ReportStarter$JasperData.class ReportStarter$ReportData.class ReportStarter.class SendByEmailListener.class SwingJRViewerProvider.class swing utils DigestOfFile.class |
lib Adempiere.jar META-INF org adempiere process compiere interfaces report swing CComboBox$1.class CComboBox$2.class CComboBox$3.class CComboBox$ReducibleKeyListener$1.class CComboBox$ReducibleKeyListener$2.class CComboBox$ReducibleKeyListener.class CComboBox$ReducibleModel.class |
lib CInstall META-INF INDEX.LIST |
lib CInstall META-INF |
lib CompiereJasper.jar META-INF MANIFEST.MF org compiere interfaces MD5.class report Barbecue.class ExportListener.class JasperReportViewer.class JasperViewer$1.class JasperViewer$2.class JasperViewer.class JRViewerProvider.class ReportStarter$FileFilter.class ReportStarter$JasperData.class ReportStarter$ReportData.class ReportStarter.class SendByEmailListener.class SwingJRViewerProvider.class utils DigestOfFile.class | |
lib CompiereJasperReqs.jar META-INF INDEX.LIST |
|
lib posterita.jar META-INF INDEX.LIST |
|
lib posteritaPatches.jar META-INF MANIFEST.MF |
|
lib webui.war build.xml RUN_build.bat |
lib webui.war |
Note that the “Adempiere.jar” on the left side contains classes not available on the right side. These classes are contained in “CompiereJasper.jar” which is only available on the right side. Note also that there is no “INDEX.LIST” available because “Adempiere.jar” is not indexed. Indexing jar files at this level is not done anymore because the files will be repacked anyway once the ADempiere setup is started.
There are some files included on the left side that simply should not be included, namely “build.xml” and “RUN_build.bat” which are part of “webui.war”. These files are not included anymore if a build is created using the cleaned up Ant build scripts.
“posterita.jar” and “posteritaPatches.jar” are not available at all on the right side. These files are not created anymore since the compiled classes of posterita are directly packaged into “posterita.war”.
Dependency Graph
Using the classpaths of the cleaned up Ant build scripts it is possible to derive a dependency graph of the ADempiere jar and war files.
As can be seen on this graph there are some redundancies. “CSTools” (probably short for “Compiere Server Tools”) and “CCTools” (probably short for “Compiere Client Tools”) contain almost the same libraries.
ADempiere setup
Current situation
In ADempiere 370LTS the build and setup processes are tightly coupled. It all starts with the invocation of “utils_dev/build.xml”. This master build script will invoke the build scripts of the individual ADempiere subprojects which will create the jar and war files. The final step done as part of this process is the creation of the build output folder in “<ADempiereRoot>/install”.
In order to install ADempiere this output folder is copied to the target machine that will actually serve as host for ADempiere. On this machine either “RUN_setup[.bat|.sh]” or “RUN_silentsetup[.bat|.sh]” will be executed. This in turn will call the class “org.compiere.install.Setup” or “org.compiere.install.SilentSetup”.
Both of these classes will then invoke Ant with “build.xml” in the ADempiere output folder. This file has originally been copied from “<ADempiereRoot>/install/Adempiere” and it will repack the jar and war files into new jar and war files as shown File:ADempiere_output_structure_2.png. Note that any customizations for ADempiere will be included as part of this step. Note that this step is actually a build step.
In addition to repacking the jar and war files on the host machine “build.xml” will create a bunch of xml and batch / shell files out of templates. These templates contain variables which are replaced with values (depending on the target environment, e.g. ports) by “build.xml”.
An example is the file “<build output directory>/jboss/server/adempiere/conf/jboss-serviceTemplate.xml” which is copied to “<build output directory>/jboss/server/adempiere/conf/jboss-service.xml”
It contains the following code (note the variable “@ADEMPIERE_JNP_PORT@”):
<attribute name="CallByValue">false</attribute> <!-- The listening port for the bootstrap JNP service. Set this to -1 to run the NamingService without the JNP invoker listening port. --> <attribute name="Port">@ADEMPIERE_JNP_PORT@</attribute> <!-- The bootstrap JNP server bind address. This also sets the default RMI service bind address. Empty == all addresses --> <attribute name="BindAddress">${jboss.bind.address}</attribute> <!-- The port of the RMI naming service, 0 == anonymous --> <attribute name="RmiPort">1098</attribute> <!-- The RMI service bind address. Empty == all addresses -->
During the Ant copy operation the variable “@ADEMPIERE_JNP_PORT@” will be replaced with the actual value of the variable which is stored in the “AdempiereEnv.properties” in the build output directory. The content of the file “AdempiereEnv.properties” is read by “build.xml” and Ant is therefore able to properly replace such variables.
Finally, after the above steps, ADempiere is ready to be started.
To emphasize: Basically the ADempiere source code is compiled and packaged and these packages are copied to the target machine where they are repackaged to the final structure. Some configuration files are created on the target machine after which ADempiere is ready to be used.
Proposed setup changes
Instead of the approach outlined in Current situation it would be preferable to do as many steps as possible on the build machine and not on the host machine. Whenever possible a setup process should be as simple as possible because it is often performed by end user and not by developers that actually know what is going on.
Preferably there shall be both a client and a server installer for both the Windows, Linux, etc. environments which would greatly simplify the actual setup and hopefully help spread ADempiere as a serious system.
In order to achieve this it is necessary to rework the existing ADempiere setup process. Instead of repackaging the jar and war files on the host machine this should already be done on the build machine. It will then be simple to write installers that will deploy these final packages.
Conclusions
The cleaned up ADempiere Ant build scripts are ready to be used. As a next step the packaging structure of ADempiere should be redesigned to remove redundancies whenever possible. Once this is done the repackaging steps, which are now done as part of the ADempiere setup, should be done as part of a build, not a setup step. This will pave the way to finally come up with clean, platform dependent installers.