Difference between revisions of "FR3409987 Cleanup of Ant build scripts"

From ADempiere
Jump to: navigation, search
This Wiki is read-only for reference purposes to avoid broken links.
Line 268: Line 268:
 
Using the classpaths of the cleaned up Ant build scripts it is possible to derive a [[:File:ADempiere_dependency_graph.png|dependency graph]] of the ADempiere jar and war files.
 
Using the classpaths of the cleaned up Ant build scripts it is possible to derive a [[:File:ADempiere_dependency_graph.png|dependency graph]] of the ADempiere jar and war files.
  
[[File:ADempiere_dependency_graph.png|thumb]]
+
[[File:ADempiere_dependency_graph.png]]
  
 
As can be seen on this graph there are some redundancies. “CSTools” (probably short for “Compiere Server Tools”) and “CCTools” (probably short for “Compiere Client Tools”) contain almost the same libraries.
 
As can be seen on this graph there are some redundancies. “CSTools” (probably short for “Compiere Server Tools”) and “CCTools” (probably short for “Compiere Client Tools”) contain almost the same libraries.
Line 278: Line 278:
 
In order to install ADempiere this output folder is copied to the target machine that will actually serve as host for ADempiere. On this machine either “RUN_setup[.bat|.sh]” or “RUN_silentsetup[.bat|.sh]” will be executed. This in turn will call the class “org.compiere.install.Setup” or “org.compiere.install.SilentSetup”.
 
In order to install ADempiere this output folder is copied to the target machine that will actually serve as host for ADempiere. On this machine either “RUN_setup[.bat|.sh]” or “RUN_silentsetup[.bat|.sh]” will be executed. This in turn will call the class “org.compiere.install.Setup” or “org.compiere.install.SilentSetup”.
  
Both of these classes will then invoke Ant with “build.xml” in the ADempiere output folder. This file has originally been copied from “<ADempiereRoot>/install/Adempiere” and it will repack the jar and war files into new jar and war files as shown [[:File:ADempiere_output_structure_2.png|here]]. Note that any customizations for ADempiere will be included as part of this step. Note that this step is actually a build step.
+
Both of these classes will then invoke Ant with “build.xml” in the ADempiere output folder. This file has originally been copied from “<ADempiereRoot>/install/Adempiere” and it will repack the jar and war files into new jar and war files as shown [[:File:ADempiere_output_structure_2.png]]. Note that any customizations for ADempiere will be included as part of this step. Note that this step is actually a build step.
  
 
In addition to repacking the jar and war files on the host machine “build.xml” will create a bunch of xml and batch / shell files out of templates. These templates contain variables which are replaced with values (depending on the target environment, e.g. ports) by “build.xml”.
 
In addition to repacking the jar and war files on the host machine “build.xml” will create a bunch of xml and batch / shell files out of templates. These templates contain variables which are replaced with values (depending on the target environment, e.g. ports) by “build.xml”.
Line 314: Line 314:
 
[[File:ADempiere_build.png|800px]]
 
[[File:ADempiere_build.png|800px]]
  
[[File:ADempiere_output_structure_2.png|800px]]
+
[[File:ADempiere_output_structure_2.png]]
  
 
===Proposed setup changes===
 
===Proposed setup changes===

Revision as of 02:33, 19 September 2011

Introduction

The central idea behind this change is to simplify and cleanup the build, installation and deployment processes of ADempiere. As a first step towards this goal the change discussed in this paper modifies the existing Ant build scripts of ADempiere. The paper then discusses additional proposed but not yet implemented changes.

Note that this wiki page is also available as pdf.

Modified Ant build scripts

ADempiere 370LTS contains the following Ant build scripts which are required to build the entire system:

    • tools/build.xml
    • base/build.xml
    • extend/build.xml
    • client/build.xml
    • JasperReports/build.xml
    • serverRoot/build.xml
    • serverApps/build.xml
    • webStore/build.xml
    • webCM/build.xml
    • sqlj/build.xml
    • posterita/posterita/build.xml
    • zkwebui/build.xml
    • install/build.xml

These project specific Ant scripts are invoked by the master build script “utils_dev/build.xml”. (Note that the scripts listed above are listed in the order they are called by the master build script)

All of the above listed build files have been modified as part of this proposed change.

Build script modifications

Compiler flags

All Java compiler calls in the Ant build scripts (via javac Ant task) now use the same arguments. To achieve this, the following properties have been added to “utils_dev/build.properties”:

compile.version=1.6
compile.debug=false
compile.optimize=true
compile.deprecation=true
compile.compress=false

"compile.version" is used to instruct the Java compiler to generate code for this version of the Java Virtual Machine (JVM).

"compile.debug" which is now set to false will prevent the compiler from adding debug information to compiled classes. Note that it is set to false since this setting shall be used to create ADempiere releases. During development it might be necessary to include debug information. In such a case this setting should be modified locally.

"compile.optimize" instructs the compiler to optimize the generated code. Note that this switch might also need to be turned off during development but shall be turned on whenever a release of ADempiere is created.

“compile.deprecation” which is set to true will show compile time warnings whenever deprecated code is used.

“compile.compress” has the effect that resulting jar, war and ear files are compressed. This will result in slightly smaller files but might have the effect that loading time increases. Because of this the default setting is “false”.

All “javac” task invocations in the Ant build scripts utilize these additional properties and typically look as follows:

<javac srcdir="${src.dir}"
       destdir="${build.dir}"
       target="${compile.version}"
       debug="${compile.debug}"
       optimize="${compile.optimize}"
       deprecation="${compile.deprecation}"
       includeAntRuntime="false"
       classpathref="project.classpath"/>

Elimination of circular dependency

The Ant build scripts contained one circular dependency:

Compiling the classes that shall be packed into “CompiereJasper.jar” requires the compiled classes which will be packed into Adempiere.jar, i.e. the compiled classes of the “client” subproject. Adempiere.jar however shall contain both the content of CompiereJasper.jar and the classes of the “client” subproject.

ADempiere circular dependency.png

So far the existing approach to build was as follows:

  1. “utils_dev/build.xml” invokes “client/build.xml” to compile the classes of the “client” subproject
  2. “client/build.xml” invokes “JasperReports/build.xml” to build the “JasperReports” subproject
  3. “client/build.xml” builds “Adempiere.jar” since both the compiled classes of the “client” and the “JasperReports” subprojects are now available
  4. “utils_dev/build.xml” needlessly invokes “JasperReports/build.xml”

Since “Adempiere.jar” will be repackaged by “install/Adempiere/build.xml” anyway there is no need for this complicated procedure.

In the cleaned up Ant build scripts “Adempiere.jar” is created before “CompiereJasper.jar” and therefore does not include the compiled classes of the “JasperReports” subproject.

“install/Adempiere/build.xml” has been modified so that “Adempiere.jar” contains the content of “JasperReports.jar” after “Adempiere.jar” has been repacked.

Exclusion of unnecessary build scripts

In addition to the Ant build scripts listed in Modified Ant build scripts the ADempiere project contained the Ant build file “JasperReportsWebApp/build.xml”. Invocation of this build script in “utils_dev/build.xml” has been removed since it does not create files that are used by ADempiere. None of the files created by “JasperReportsWebApp/build.xml” are referenced anywhere nor are they packed into the ADempiere jar, war or ear files.

Classpath

The classpath for all modified Ant build scripts has been minimized, i.e. all unnecessary entries in the classpaths of the scripts have been removed.

Creation of output structure

The output folder containing the ADempiere build has so far been created by the Ant build script “install/build.xml”. This functionality has been moved to the master build file “utils_dev/build.xml”. This has been done because this seems to be a more logical place to perform this.

Moved files

The Ant build file “posterita/posterita/build.xml” has been moved to “posterita/build.xml”.

Note that the ADempiere output folder is now located in “<ADempiereRoot>/build” instead of “<ADempiereRoot>/install”.

General

In general it has been attempted to make the structure of the build scripts as straightforward as possible. All modified build scripts use the same structure, target names, etc.

Diff between old and new build output

The goal of that first cleanup was to keep the ADempiere build output structure as similar to the already existing structure as possible. Nonetheless there are some differences which are listed in the following table. The table lists all orphans, i.e. files that are only available on one side.

This list has been created by a compare tool (“Beyond Compare”) and it includes the content of all jar and war files. The left side lists all files of the ADempiere 370LTS release that are not available in a build created using the cleaned up Ant build scripts and vice versa.


Released ADempiere 370LTS ADempiere 370LTS created using the modified scripts
 lib
   Adempiere.jar
     META-INF
       INDEX.LIST
     org
       adempiere
         process
           DBA_PrepareRelease.class
       compiere
         interfaces
           MD5.class
         report
           Barbecue.class
           ExportListener.class
           JasperReportViewer.class
           JasperViewer$1.class
           JasperViewer$2.class
           JasperViewer.class
           JRViewerProvider.class
           ReportStarter$FileFilter.class
           ReportStarter$JasperData.class
           ReportStarter$ReportData.class
           ReportStarter.class
           SendByEmailListener.class
           SwingJRViewerProvider.class
         swing
           
           
           
           
           
           
           
         utils
           DigestOfFile.class
 lib
   Adempiere.jar
     META-INF
       
     org
       adempiere
         process
         
       compiere
         interfaces
           
         report
           
           
           
           
           
           
           
           
           
           
           
           
           
         swing
           CComboBox$1.class
           CComboBox$2.class
           CComboBox$3.class
           CComboBox$ReducibleKeyListener$1.class
           CComboBox$ReducibleKeyListener$2.class
           CComboBox$ReducibleKeyListener.class
           CComboBox$ReducibleModel.class
           
           
 lib
   CInstall
     META-INF
       INDEX.LIST
 lib
   CInstall
     META-INF
     
 lib
   CompiereJasper.jar
     META-INF
       MANIFEST.MF
     org
       compiere
         interfaces
           MD5.class
         report
           Barbecue.class
           ExportListener.class
           JasperReportViewer.class
           JasperViewer$1.class
           JasperViewer$2.class
           JasperViewer.class
           JRViewerProvider.class
           ReportStarter$FileFilter.class
           ReportStarter$JasperData.class
           ReportStarter$ReportData.class
           ReportStarter.class
           SendByEmailListener.class
           SwingJRViewerProvider.class
         utils
           DigestOfFile.class
 lib
   CompiereJasperReqs.jar
     META-INF
       INDEX.LIST
 lib
   posterita.jar
     META-INF
       INDEX.LIST
 lib
   posteritaPatches.jar
     META-INF
       MANIFEST.MF
 lib
   webui.war
     build.xml
     RUN_build.bat
 lib
   webui.war
   
   

Note that the “Adempiere.jar” on the left side contains classes not available on the right side. These classes are contained in “CompiereJasper.jar” which is only available on the right side. Note also that there is no “INDEX.LIST” available because “Adempiere.jar” is not indexed. Indexing jar files at this level is not done anymore because the files will be repacked anyway once the ADempiere setup is started.

There are some files included on the left side that simply should not be included, namely “build.xml” and “RUN_build.bat” which are part of “webui.war”. These files are not included anymore if a build is created using the cleaned up Ant build scripts.

“posterita.jar” and “posteritaPatches.jar” are not available at all on the right side. These files are not created anymore since the compiled classes of posterita are directly packaged into “posterita.war”.


Dependency Graph

Using the classpaths of the cleaned up Ant build scripts it is possible to derive a dependency graph of the ADempiere jar and war files.

ADempiere dependency graph.png

As can be seen on this graph there are some redundancies. “CSTools” (probably short for “Compiere Server Tools”) and “CCTools” (probably short for “Compiere Client Tools”) contain almost the same libraries.

ADempiere setup

Current situation

In ADempiere 370LTS the build and setup processes are tightly coupled. It all starts with the invocation of “utils_dev/build.xml”. This master build script will invoke the build scripts of the individual ADempiere subprojects which will create the jar and war files. The final step done as part of this process is the creation of the build output folder in “<ADempiereRoot>/install”.

In order to install ADempiere this output folder is copied to the target machine that will actually serve as host for ADempiere. On this machine either “RUN_setup[.bat|.sh]” or “RUN_silentsetup[.bat|.sh]” will be executed. This in turn will call the class “org.compiere.install.Setup” or “org.compiere.install.SilentSetup”.

Both of these classes will then invoke Ant with “build.xml” in the ADempiere output folder. This file has originally been copied from “<ADempiereRoot>/install/Adempiere” and it will repack the jar and war files into new jar and war files as shown File:ADempiere_output_structure_2.png. Note that any customizations for ADempiere will be included as part of this step. Note that this step is actually a build step.

In addition to repacking the jar and war files on the host machine “build.xml” will create a bunch of xml and batch / shell files out of templates. These templates contain variables which are replaced with values (depending on the target environment, e.g. ports) by “build.xml”.

An example is the file “<build output directory>/jboss/server/adempiere/conf/jboss-serviceTemplate.xml” which is copied to “<build output directory>/jboss/server/adempiere/conf/jboss-service.xml”

It contains the following code (note the variable “@ADEMPIERE_JNP_PORT@”):

<attribute name="CallByValue">false</attribute>
<!-- The listening port for the bootstrap JNP service. Set this to -1
   to run the NamingService without the JNP invoker listening port.
-->
<attribute name="Port">@ADEMPIERE_JNP_PORT@</attribute>
<!-- The bootstrap JNP server bind address. This also sets the default
   RMI service bind address. Empty == all addresses
-->
<attribute name="BindAddress">${jboss.bind.address}</attribute>
<!-- The port of the RMI naming service, 0 == anonymous -->
<attribute name="RmiPort">1098</attribute>
<!-- The RMI service bind address. Empty == all addresses
-->

During the Ant copy operation the variable “@ADEMPIERE_JNP_PORT@” will be replaced with the actual value of the variable which is stored in the “AdempiereEnv.properties” in the build output directory. The content of the file “AdempiereEnv.properties” is read by “build.xml” and Ant is therefore able to properly replace such variables.

Finally, after the above steps, ADempiere is ready to be started.

To emphasize: Basically the ADempiere source code is compiled and packaged and these packages are copied to the target machine where they are repackaged to the final structure. Some configuration files are created on the target machine after which ADempiere is ready to be used.

ADempiere build.png

ADempiere output structure 2.png

Proposed setup changes

Instead of the approach outlined in Current situation it would be preferable to do as many steps as possible on the build machine and not on the host machine. Whenever possible a setup process should be as simple as possible because it is often performed by end user and not by developers that actually know what is going on.

Preferably there shall be both a client and a server installer for both the Windows, Linux, etc. environments which would greatly simplify the actual setup and hopefully help spread ADempiere as a serious system.

In order to achieve this it is necessary to rework the existing ADempiere setup process. Instead of repackaging the jar and war files on the host machine this should already be done on the build machine. It will then be simple to write installers that will deploy these final packages.

Conclusions

The cleaned up ADempiere Ant build scripts are ready to be used. As a next step the packaging structure of ADempiere should be redesigned to remove redundancies whenever possible. Once this is done the repackaging steps, which are now done as part of the ADempiere setup, should be done as part of a build, not a setup step. This will pave the way to finally come up with clean, platform dependent installers.