Lazy loading Trees

From ADempiere
Revision as of 03:28, 21 December 2012 by Jan Thielemann (Talk) (Testcases)

Jump to: navigation, search
This Wiki is read-only for reference purposes to avoid broken links.

Lazy loading Trees - Functional Specifications

Status

User, Date Status Description Points
Mo 11:15, 17. Dec. 2012 (CEST) In Development First development iteration. Implementation of lazy loading tree nodes. Loading trees with some million nodes on several levels is now possible.There are still some todos in the current code.
Mo 08:23, 20. Dec. 2012 (CEST) In Development Second development iteration. Cleaner code, buxfixes, improvements, more stability.
Mo 11:42, 12. Jul. 2010 (CEST) Technical Team: Approval ... Further Information ...
Mo 14:46, 23. Nov. 2010 (CET) Functional Team: Approval ... Further Information ...
Ts 18:55, 26. Nov. 2010 (CET) Other Status ... ... Further Information ...

Contributors

This feature is contributed by evenos GmbH

Overview

Currently Trees in adempiere are implemented not very well. When a tree is loaded (e. g. via VTreePanel in the swing client or via ADTreePanel in the webui), the whole tree is loaded at once by MTree.java. With lazy loading, a tree should only load a node when it needs the node. This means, if only the first level of the tree is shown, the second level is not loaded now. Only if a first level node is opened, the second level nodes for only this node are loaded. Also most of the loading behaviour should be placed in the nodes itselfs or in the model, not in MTree.

Purpose

The purpose for this development is simple: it's needed. Imagin a tree with 100 noded. Each node has 100 subnodes itself and each of the subnodes has again 100 subnodes. Together, in this tree there are 1,000,000 nodes. Adempiere would not only need to query the million nodes from the database, it also would need to create a million MTreeNode objects. This will probably take some time. In fact, if you try to create a tree like this and open it with adempiere, the swing client will be not responsive for several minutes. The webui will possible throw an error like "Java heap space".

In this feature, we will refactore the whole tree handling without losing functionality.

Miscellaneous

Some parts of the tree functionality didn't work as expected. We fixed some of these issues including:

  • Removing sections from the Favourites Bar when no node is left in this section
  • Show Tooltips with the node description when hovering with the mouse over a node
  • Searching in Trees now makes use of the name AND the description of a node
  • Dropping a node on one of its children or itself is not possible anymore
  • Deleting a summary node moves all its children to its parent instead of removing the nodes from the tree. This is a better solution because otherwise there would be entries in the grid which never show up in the tree!
  • Pasting the same node multiple times is not possible anymore because we clear the clipboard after the paste
  • Moving a node now saves the correct seqno (see https://groups.google.com/forum/#!topic/idempiere/ETF7fymVZRM 2nd post)

Design Considerations

Assumptions

  • A Node is loaded when it is needed
  • No more loading the whole tree at once
  • Cleaner code / responsibilities
  • Cleaner seperation between Swing Client / Webui

Dependencies

None. This feature should work out of the box by replacing the changed files. There are no other/new dependencies.

Constraints

  • There is some code which may only work with PostgreSQL (SQL: "with recursive...") which need some adjustment to work with oracle and mysql (probably in the "convert" classes).
  • Searching in the tree is not top to bottom anymore but random due to the problem that we don't have the whole tree loaded which we can search from top to bottom
  • A node can still be cut (ctrl+x) even if there is already a node in the clipboard. This causes the previous cutted node to disappear (till cache is resetted or user log out and in again which causes the tree to reload)
  • A node can be dragged or pasted into trees of other windows

Glossary

Functional Requirements

There are no functional requirements. Just replace the changed files.

Acceptance criteria

QA and test cases

These Trees should work like before and as expected.

Prerequisites

First we need a really big tree so we can see what the problem is with the default behaviour compared to our lazy implementation. We decided to use the Organization since we don't need much information to create a tree there.

Run the following script on your database (PostgreSQL) to create to create a tree with 100 nodes each level and 3 levels:

--generate toplevel nodes
INSERT INTO adempiere.ad_treenode (ad_tree_id, node_id, ad_client_id, ad_org_id, isactive, created, createdby, updated, updatedby, parent_id, seqno)
SELECT 50, 2000000 + y.id, 0, 0, 'Y', now(), 0, now(), 0, 0, 999
FROM generate_series(1,99) AS y(id);

--generate subnodes
INSERT INTO ad_treenode (ad_tree_id, node_id, ad_client_id, ad_org_id, isactive, created, createdby, updated, updatedby, parent_id, seqno)
SELECT 50, 2000000 + y.id + x.id*100, 0, 0, 'Y', now(), 0, now(), 0, 2000000 + x.id, 999
FROM generate_series(1,99) AS x(id), generate_series(1,99) AS y(id);

--generate subsubnodes
INSERT INTO ad_treenode (ad_tree_id, node_id, ad_client_id, ad_org_id, isactive, created, createdby, updated, updatedby, parent_id, seqno)
SELECT 50, 2000000 + y.id + x.id*100 + z.id*10000, 0, 0, 'Y', now(), 0, now(), 0, 2000000 + y.id + x.id *100 , 999
FROM generate_series(1,99) AS x(id), generate_series(1,99) AS y(id), generate_series(1,99) AS z(id);

--generate toplevel organizations
INSERT INTO ad_org
SELECT 2000000 + y.id , 0, 'Y', now(), 100, now(), 100, 2000000 + y.id, 2000000 + y.id, 2000000 + y.id, 'Y', NULL
FROM generate_series(1,99) AS y(id);

--generate suborganizations
INSERT INTO ad_org
SELECT 2000000 + y.id + x.id*100, 0, 'Y', now(), 100, now(), 100, 2000000 + y.id + x.id*100, 2000000 + y.id + x.id*100, 2000000 + y.id + x.id*100, 'Y', NULL
FROM generate_series(1,99) AS x(id), generate_series(1,99) AS y(id);

--generate subsuborganizations
INSERT INTO ad_org
SELECT 2000000 + y.id + x.id*100 + z.id*10000, 0, 'Y', now(), 100, now(), 100, 2000000 + y.id + x.id*100 + z.id*10000, 2000000 + y.id + x.id*100 + z.id*10000, 2000000 + y.id + x.id*100 + z.id*10000, 'Y', NULL
FROM generate_series(1,99) AS x(id), generate_series(1,99) AS y(id), generate_series(1,99) AS z(id);

--generate toplevel orginfo
INSERT INTO ad_orginfo (ad_org_id, ad_client_id, isactive, created, createdby, updated, updatedby, duns, taxid)
SELECT 2000000 + y.id, 0, 'Y', now(), 100, now(), 100, 0, 0
FROM generate_series(1,99) AS y(id);

--generate suborginfo
INSERT INTO ad_orginfo (ad_org_id, ad_client_id, isactive, created, createdby, updated, updatedby, duns, taxid)
SELECT 2000000 + y.id + x.id*100, 0, 'Y', now(), 100, now(), 100, 0, 0
FROM generate_series(1,99) AS x(id), generate_series(1,99) AS y(id);

--generate subsuborginfo
INSERT INTO ad_orginfo (ad_org_id, ad_client_id, isactive, created, createdby, updated, updatedby, duns, taxid)
SELECT 2000000 + y.id + x.id*100 + z.id*10000, 0, 'Y', now(), 100, now(), 100, 0, 0
FROM generate_series(1,99) AS x(id), generate_series(1,99) AS y(id), generate_series(1,99) AS z(id);

If you want to delete all the nodes/organizations later, just run:

delete from ad_treenode where node_id > 2000000;
delete from ad_org where ad_org_id > 2000000;


Testcases

Task Expectation Result Status
Swing Client - Main Window Menu Tree ------------------------------------ ------------------------------------ -----
Start Adempiere and log in with different users/roles Tree is trimmed and shows only nodes which the current user/role can access. Empty summary folders are removed from the tree. The favourites bar shows the correct nodes for the current user/role
Single click on a node in the tree If the node is a summary node (folder), the node gets selected. If the node is a window, task, workflow, report or process, the selected window, taks,..., is opened/started
Double click on a node in the tree If the node is a summary node (folder), the node gets opened/closed. If the node is a window, task, workflow, report or process, the selected window, ..., is opened/started
Right click on a node in the tree A popup menu shows up. If node is a folder, the menu entry in the popup menu is disabled. If node is not a folder, the node can be added to the favourites bar. A node can only be added once. Adding a node two times gives the user a message that the node is already in the bar. A node is added to the section of its first level parent (e. g. System Admin > General Rules > System Rules > System is added to the section System Admin)
Check/Uncheck the Expand Tree checkbox The tree expands/collapses to show all or only the first level nodes
Type in some text in the Lookup field and hit return Every time the user hits return, the next found node with the searchtext in its name or description gets selected and shown to the user. The order in which nodes are found is various but a node is first found a second time after all other nodes are selected once. E. g. if we have nodes 1, 11, 1a1 and 2 and we search for "1", the order might be 1, 1a1, 11, 1, 1a1, 11... or 11, 1a1, 1, 11, 1a1, 1...
Single click on a node in the favourites bar The correct windows, task, ..., is opened/started
Right click on a node in the favourites bar A popup menu shows up. With the menu entry in this popup menu it is possible to remove the selected node from its section
Remove the last node from a section in the favourites bar If no nodes are left in a section, the whole section is removed from the favourites bar
Drag and Drop a node or cut/paste it via ctrl+x/ctrl+v Drag and Drop or cut/paste must not work on the main menu tree
Swing Client - Editable Tree in any Window ------------------------------------ ------------------------------------ -----
Single click on a node in the tree The node and the corresponding entry in the grid/single row layout gets selected.
Double click on a node in the tree Same as single click. If node is a summary folder, the folder opens/closes
Right click on a node in the tree A popup menu with two entries shows up. Selecting one of the menu entires either cuts a node or pastes it.
Cut a node Invoked by either selecting the "Move Item" menu entry from the popup or by selecting a node and press ctrl+x. The previous selected node gets removed from the tree and is now in the systems clipboard, ready to be pasted at another place
Paste a node Invoked by either selecting the "Insert Item here" menu entry from the popup or by selecting a node and press ctrl+v. The node from the clipboard is now inserted after the selected node if node is not a summary folder. If node is a summary folder, the node from the clipboard gets inserted as the first child.
Cut a node when there is already a node in the clipboard The previous cutted node gets inserted at it's previous position and the new cutted node is removed from the tree and gets stored to the clipboard Cutting a second node causes the previous cutted node to be still removed from the tree. After a cache reset or log out and in again the node is back in place does not work
Drag and Drop a node The selected node can be moved to the desired position. If the node is moved onto a summary folder, it is inserted as the first child of this folder.
Change the name, description, action or summary level

Development infrastructure

Technical Requirements

Data Requirements

Non-Functional Requirements

Open Discussion Items

Closed Discussion Items