PMC Architecture Meeting 20100317

From ADempiere
Jump to: navigation, search
This Wiki is read-only for reference purposes to avoid broken links.

Date: 2010-03-17
Time: 7AM GMT
Venue: irc #adempiere-team
Support Spreadsheet: Adempiere PMC Architecture
Chat times in GMT-5

Agenda

  • Wish #8/#9
    • What are the concrete requirements to Dictionary Audit?
    • Who wants to sketch a solution?
  • OSGi discussion
    • Deployment: Retain old deployment, move to new? (Tomcat? - Webstart JNLP generation? - Windows installer?)
    • OSGi metadata vs AD? (pointers to code in AD or pointers to AD in code or both?)
    • About branch handling: Introducing plugins is about code reorganisation, which is hard to do in a branch - how to deal with that?
  • Continue Review Wishlist
    • only priority high, per topic, starting at #11:
      • author shortly explains topic
      • others agree on priority
      • discussion on first tasks for the topic, put them on the task list
  • Opinions about unplugging unmaintained or underdeveloped extensions from core (CM, HTML UI, Posterita)
  • Other items?

Summary

  • #8/#9: Dictionary audit
    • Big subproject
    • Needs to but prototyped
    • Dictionary entries: Core UNION Module entries
    • Model classes: Proxied in OSGi
  • OSGi open questions:
    • EJBs? Cann they called by OSGi plugins? Can they use OSGi plugins?
    • Web store and admin?
    • All in the same OSGi container?
    • Metadata: AD vs. Manifest: In AD is mandatory, optionally also in Manifest
  • Mercurial:
    • No objections, only timeframe a bit long?

Chat

(02:01:18) CarlosRuiz: starting now Architectural
(02:01:26) nwessel: cu
(02:01:54) viola: OK, agenda as usual here: http://www.adempiere.com/index.php/PMC_Architecture_Meeting_20100317
(02:02:28) viola: Carlos: Sorry again for removing the calendar entries last week - trying to learn ical... :-(
(02:02:50) CarlosRuiz: no problem
(02:02:50) hengsin: hi Jorg
(02:02:55) CarlosRuiz: fortunately it was not google  :-)
(02:03:01) viola: hehe
(02:03:37) viola: so... last time we ended with #8/#9: dictionary audit
(02:04:14) CarlosRuiz: about #8 - do you want to discuss it or just to clarify the point?
(02:04:48) viola: hmm - i think point is clear
(02:04:54) viola: solution not ;-)
(02:06:18) trifon: what means dictionary audit?
(02:06:20) CarlosRuiz: maybe we can maintain 2pack or backward compatibility
(02:06:20) CarlosRuiz: and put hengsin modifications in trunk as 3pack  :-)
(02:06:24) hengsin: one significant requirements is here to managed the modifications by extension and customization
(02:06:29) trifon: ability to see who, when changed records?
(02:07:06) viola: trifon yes - and uninstall
(02:07:28) trifon: ok. OSGI provide this infrastructure, right?
(02:07:37) viola: not really
(02:07:41) trifon: OSGI has dependency between modules.
(02:07:44) hengsin: at the moment, official dictionary entries can be overriden by customizaton and migration script can silently remove your customization too.
(02:07:44) trifon: ups, why?
(02:08:04) viola: OSGi provides code organsiation, plugin install and such
(02:08:14) viola: and automatic dictionary upgrade via 2pack
(02:08:25) viola: but no audit nor uninstall
(02:08:35) trifon: viola: yes. right. OSGi is the infrastructure.
(02:08:38) trifon: i see.
(02:08:57) viola: if we could upgrade 2pack to 3pack that handles that
(02:09:05) viola: OSGi could make use of that
(02:09:08) trifon: but eclipse has uninstall. i supose they do it with p2 their
(02:09:26) viola: yes you can unstanll the code, the plugin
(02:09:33) viola: but what about the dictionary changes ?
(02:09:54) viola: We can decide in OSGi that a plugin goes away
(02:10:11) viola: but we have to rollback the changes it made to the dictionary
(02:10:37) viola: I would feel better if we could factor that out of OSGi as 2/3pack
(02:10:50) hengsin: I guess we can either rely on the change log to audit that or we can mandate that customization/extensions cannot update other "entity type" entries.
(02:11:16) trifon: hengsin: this limits usability.
(02:11:48) viola: 2pack already has a backup log
(02:11:48) trifon: i mean then your customization can't edit ademepire core entries.
(02:12:16) hengsin: trifon: we can provide an extend mechanism, for e.g, ad_column_ext ...
(02:12:17) viola: hengsin: can we implement a rollback based on this?
(02:12:32) hengsin: Jorg: yes, it is partially implemented
(02:12:58) viola: if we have ad_column_ext which a column call "module"
(02:13:13) hengsin: the rollback is also partially implemented in adempiere for ad_changelog.
(02:13:14) viola: what if there are two entries for two modules for the same column?
(02:13:30) viola: i.e. how to handle conflicts?
(02:13:51) viola: (this is #9)
(02:14:03) hengsin: Jorg: they will be using different entity type. so it could be either the last one win or we could block the installation
(02:14:10) trifon: it getting more complicated.
(02:14:29) hengsin: trifon: it is now, just that we pretend it is not.
(02:14:55) viola: hengsin: diffferent entitiy type means? - which entry is used for UI generation?
(02:15:03) viola: i think blocking is correct
(02:15:31) hengsin: Jorg: it should be ad_column union ad_column_ext ...
(02:15:32) trifon: have to go now. will be back soon.
(02:15:38) trifon has left the channel (quit: Quit: ChatZilla 0.9.86 [Firefox 3.5.8/20100214235958]).
(02:16:06) viola: hmm say module 1 changes column length to 15, module 2 to 10 - ...?
(02:16:42) CarlosRuiz: that's an easy conflict - keep the longest
(02:16:42) CarlosRuiz: the problem is when they overwrite reference, or callout
(02:16:55) viola: carlos ok
(02:16:57) hengsin: Jorg: if last one win, it will be 10, if we do blocking, then module 2 can't be install as it cause dictionary conflict
(02:17:33) viola: ok - then i understand
(02:17:34) hengsin: and as carlos mention, there can be additional rules here, like extend is not a problem, etc ...
(02:18:03) viola: ok thats for conflict - back to uninstall
(02:18:22) viola: this is solved by your union
(02:18:38) viola: ... duplicated entries: last one wins - correct?
(02:18:50) viola: then module only has to remove its entry
(02:18:55) hengsin: yes, jorg, if we use union and each module use different entitytype then uninstall is straigth forward
(02:19:04) viola: i see
(02:19:22) viola: and this for each dictionary entitiy
(02:19:37) viola: ... translation included
(02:20:08) hengsin: yes jorg, thats basically how xml type configuration work, each module keep their own meta data
(02:20:35) CarlosRuiz: we're still in #9 - I think that's a big elephant - we need to split it in minor pieces to make it digestible
(02:20:35) CarlosRuiz: sounds like a subproject
(02:21:16) hengsin: yes and we will need to prototype any idea that we think is worth pursuing
(02:21:52) viola: and now comes the question: Who is doing that?
(02:21:55) hengsin: actually, there is another conflict that we need to resolve - the model class
(02:22:33) viola: yepp
(02:22:44) viola: model class are an issue in OSGi too
(02:23:03) hengsin: I'm thinking that customization/extension should not override core model class but we provide a tool to generate interface and dynamic proxy adapter
(02:23:12) hengsin: and they can used that when it is needed.
(02:23:15) viola: two plugins, different - non-conflicting - additional attributes - which model class
(02:23:29) CarlosRuiz: or, extensions must use the generic setters / getters
(02:23:56) viola: i hate dyn proxies (currently struggling with them in my project)
(02:23:58) hengsin: Carlos: yes, that works but 1. difficult to enfore, 2. not developer friendly
(02:24:17) viola: therefore i like carlos proposal though i see hengsins points too
(02:24:36) viola: so: Each plugin defines its own class extending core class
(02:24:52) hengsin: since model class is just a wrapper for the generic getter / setter, we can have tools that generate interface for a module and they can used that with the help of dynamic proxy
(02:24:56) viola: but then they always have to be loaded again from db
(02:25:03) hengsin: no Jorg, just interface and dynamic proxy
(02:25:08) CarlosRuiz: agree with 2 - 1 is easy to enforce - not allow overwriting a core class on extension
(02:25:40) hengsin: Jorg, don't have to you can use dynamic proxy to adapt an model class
(02:25:49) hengsin: to the interface you want
(02:26:09) viola: hengsin yes
(02:26:27) viola: i just fear the proxy introduces probs
(02:26:48) hengsin: Jorg: performance issue ?
(02:26:53) viola: i can say for example that drools - whenever we want to intruduce rules engines - cannot handle them
(02:26:57) hengsin: like PermGen outofmemory :
(02:27:42) viola: no hengsin - in 1.6 proxies do free permgen after use (as far as i know)
(02:27:51) hengsin: anyway, if your agree that this is a good idea, I can try to prototype this ...
(02:28:10) viola: ok - for this i'll volunteer
(02:28:25) viola: oh sorry didn't read correctly
(02:28:32) viola: certainly you can!
(02:29:33) viola: so we have a result: In OSGi model classes will be proxied and hengsin tries that
(02:30:01) viola: in 2pack (and AD generally) we need module-specific changes (no volunteer until now)
(02:30:21) viola: this addresses #8 and #9 i think?
(02:30:57) hengsin: carlos, anything to add ?
(02:31:51) CarlosRuiz: sorry I'm behind with architectural group - solving a crisis in functional at this moment (bugs, etc(
(02:31:51) CarlosRuiz: so, I'm really happy with you two, and I try to not interfere as I'm not being too much helpful
(02:32:21) viola: hengsin: added a task for you - leaving expected date open ;-)
(02:32:33) hengsin: ok, no problem
(02:32:47) hengsin: lets move on to the OSGi work
(02:33:05) viola: ok i have three questions here
(02:33:18) viola: 1. Deployment: Retain old deployment, move to new?
(02:33:32) hengsin: for the osgi deployment, does it means we need no application server since servlet engine is already part of the osgi stack ?
(02:33:59) viola: no, i currrntly simply throw i all into a plain tomcat
(02:34:08) viola: works (despite ejb)
(02:34:20) hengsin: hmm .. isn't jetty since is there ?
(02:34:34) viola: right
(02:34:38) hengsin: sorry, i means isn't jetty servlet engine is bundle with equinox ?
(02:34:59) hengsin: so why do we still need tomcat then ?
(02:35:09) viola: u are talking about that "servlet container inside OSGi" here
(02:35:21) viola: i didn't want that
(02:35:33) viola: because we then would be tied into jetty and equinox
(02:35:36) hengsin: hmm ... what's the problem with that ?
(02:35:55) viola: instead i used the "OSGi inside servlet container" thing
(02:36:00) viola: with the servlet bridge
(02:36:03) viola: works
(02:36:17) viola: and is able to work in any servlet/j2ee container
(02:36:39) hengsin: I just wondering is the other way round looks much simpler deployment wise.
(02:36:51) viola: simpler maybe
(02:37:17) viola: but if you alyready have a deployment environment - its a big move
(02:37:37) viola: not simple
(02:37:50) viola: you have a war and throw it into tomcat
(02:38:07) viola: or jetty or jboss
(02:38:43) hengsin: ok, I guess you means it is more flexible this way and more compatible with what people have been doing
(02:38:52) viola: right!
(02:39:09) hengsin: alright, if we follow this line, it means we would create a separate ear file for the deployment
(02:39:20) hengsin: I means for the ejb deployment
(02:39:45) hengsin: and do away with the current jboss bundle practise ?
(02:39:45) viola: yes
(02:40:06) CarlosRuiz: one question from my ignorance - with the move to OSGi - will Adempiere still run standalone like today? I mean it can be run without a server
(02:40:25) viola: yepp - maybe for the jun 14 release we should try a parallel deployment
(02:40:45) hengsin: carlos, yes just like eclipse can run without application server
(02:41:00) hengsin: I means eclipse is osgi based
(02:41:02) CarlosRuiz: great
(02:41:42) hengsin: Jorg, that mean we will need to rework the installation tool to cater for external application server - 1 for web and 1 for ejb
(02:41:59) hengsin: although the 2 could be the same but they doesn't have to
(02:42:02) viola: why?
(02:42:08) viola: ah ok
(02:42:11) viola: exactly
(02:42:36) viola: the installation tool in first step simply should build a new web and webstart archive
(02:42:46) viola: and put it beside the "old" ear
(02:43:12) hengsin: it is more than that, we still have webstore and the admin apps
(02:43:15) CarlosRuiz: #29 -> replace ejb ?
(02:43:26) hengsin: I think the admin apps could be broken by this move too
(02:43:51) viola: i have to look into that
(02:43:58) CarlosRuiz: and as webapps we have posterita and webservices
(02:44:00) trifon [~chatzilla@pD95F0BD9.dip0.t-ipconnect.de] entered the channel.
(02:44:01) hengsin: carlos, I think that's optional follow Jorg's proposed approach. at least it is not significant
(02:44:38) viola: i also would try to postpone the ejb thing
(02:44:49) CarlosRuiz: ok
(02:44:58) viola: simply try out whether they are reachable from an OSGi install
(02:45:16) hengsin: Jorg: that shouldn't be an issue
(02:45:43) viola: so we have an answer to 1: Stay with JBoss and deploy a new WAR
(02:46:08) viola: (only i would try to change the dev environment because tomcat is so much handier than JBoss...)
(02:46:30) viola: Question 2: OSGi metadata vs AD?
(02:46:38) viola: Please no wars about that ;-)
(02:46:49) CarlosRuiz: what is jboss requirement from adempiere?
(02:46:50) hengsin: hmm .. Jorg, forgot one thing here - wouldn't it be an issue as we don't have the equivalent of servlet-bridge for ejb ?
(02:46:59) CarlosRuiz: I mean - what do we need from jboss that tomcat doesn't have?
(02:47:03) trifon: viola: by OSGi metadata is meant MANIFEST.MF, right?
(02:47:09) trifon: all the data stored inside?
(02:47:19) viola: wait - back to question 1 - sorry
(02:47:43) viola: hengsin: dont think so - let me try it out ok?
(02:47:53) viola: carlos: EJB support
(02:48:27) hengsin: ok, we need to find that out - if that's an issue then we must move to web services.
(02:48:37) viola: yepp
(02:49:16) hengsin: trifon: yes, manifest.mf and plugin.xml
(02:49:18) viola: but at least remote interfacing should work
(02:49:26) viola: so now to question 2
(02:49:46) CarlosRuiz: and ejbs in adempiere are just for db connection discovery and accounting posting (almost deprecated)
(02:49:48) hengsin: jorg: yes but if it can't use the osgi module then it is a no go
(02:50:22) viola: hengsin didn't get that?
(02:50:59) viola: u mean if EJBs cannot use OSGi?
(02:51:11) hengsin: for e.g, ejb will call the accounting processor and that might need to lookup posting rules extension from osgi module
(02:51:19) viola: I see
(02:51:44) viola: OK I'll have this in my mind when testing it
(02:52:35) hengsin: yes and the osgi instance area should be shared by all web module too ( zk, posterita, webservice and webstore )
(02:53:32) CarlosRuiz: > Question 2: OSGi metadata vs AD? Please no wars about that ;-)
(02:53:32) CarlosRuiz: I would bet for a mixed environment (more work, but the best of both worlds), having possibility to have AD total flexibility - and when an implementation is stable it can move to metadata for speed up the thing
(02:53:32) CarlosRuiz: But, if not possible, I prefer flexibility over speed (unless the speed penalty is prohibitive)
(02:54:02) viola: hengsin but this is always configurable and thus no prob - though you want diffenrent bundles (classloaders for different web apps right?
(02:54:24) trifon: here open question i think is: can we have all meatadata which we have now in AD into OSGi?
(02:54:34) trifon: i would say this generaly this is possible, right?
(02:54:37) hengsin: yes, some share, some exclusive, that's the idea
(02:54:52) hengsin: trifon, yes, we have already done that for callout
(02:55:00) trifon: hengsin: i know.
(02:55:20) trifon: ok then open question is what metadata from AD would be good to be moved to ISGI?
(02:55:28) hengsin: carlos: the option here is either we duplicate that or we can have ad entry that point to osgi extensions
(02:55:29) trifon: OSG.
(02:55:48) viola: the only argument for metadata in OSGi is simpler development and deployment
(02:55:52) hengsin: Jorg, right ?
(02:56:14) trifon: second argument is that all metadata is stored on one place.
(02:56:19) viola: hengsin yes
(02:56:30) trifon: and you can simpley remove osgi module then metadata is gone too.
(02:56:58) viola: trifon to be honest
(02:57:07) viola: i simply think that move is much too big
(02:57:13) CarlosRuiz: hmmm - I guess I was talking about something different then
(02:57:51) hengsin: Jorg: what's your idea ?
(02:58:09) viola: at the moment i would leave all meta data in AD
(02:58:16) hengsin: I saw you have try both in approach 1 and 2 ...
(02:58:25) viola: and provide the plugin developer with tool
(02:58:27) CarlosRuiz: do you mean?
(02:58:27) CarlosRuiz: we can define a callout in AD_Column at this moment
(02:58:27) CarlosRuiz: but with OSGi we can define the callout in metadata of the plugin - and not in AD_Column
(02:58:27) CarlosRuiz: right?
(02:58:46) viola: carlos right
(02:59:09) viola: but as hengsin said: Then you cannot see that in the AD
(02:59:25) hengsin: Jorg, isn't it the combination of both at the moment ? i.e it first get from OSGi, then get from AD_Column ?
(02:59:40) trifon: CarlosRuiz: at this moment OSGi is also creating records into AD. so you have the metadata in two places: OSGi bundle and AD>
(02:59:41) viola: hengsin at the moment for callout yes
(03:00:29) CarlosRuiz: sounds interesting - because we have a big problem with callout that can be solved that way
(03:00:29) CarlosRuiz: we're thinking to add a new table AD_Column_Callout where each record is a callout
(03:00:29) CarlosRuiz: but with OSGi we could solve it in metadata
(03:00:33) viola: trifon: That finally should not be the idea - you agree?
(03:00:56) trifon: viola: yes. agree. Metadata must be at one place.
(03:01:12) hengsin: ok, so 1) to do it like how callout works now in the osgi branch 2) drop AD 3) have osgi:xxx entries in AD
(03:01:22) hengsin: Jorg, one of the three, right ?
(03:01:35) viola: ahm 2 sound a bit funny
(03:02:12) viola: we only speak about the points where AD and code has to be connected - right?
(03:02:20) viola: callout, process, validator, ...
(03:02:28) hengsin: Jorg, yes
(03:03:01) viola: yes then 1 or and drop the corresponding AD column or 3
(03:03:48) hengsin: Jorg: since you have try 1 & 3, what do you proposed here ?
(03:03:53) viola: if metadata should be at one place: 3
(03:04:17) viola: i think this is the solution and we provide tools for developers to simply and safely
(03:04:24) viola: add these AD entries
(03:04:40) viola: (as auto 2pack packout eg)
(03:04:53) viola: trifon?
(03:05:53) interopen: Hi all, i also see better to keep metadata in one place, option 3
(03:06:11) trifon: viola: i'm a bit confused.
(03:06:21) hengsin: I like 1, it make osgi module development much simpler :)
(03:06:26) trifon: if we keep metadata at one place why we need to create enitrees in AD?
(03:06:49) viola: the idea would be: At module installation
(03:07:10) viola: the module creates these entries with special markers "osgi://MyCallout"
(03:07:26) viola: at runtime, they are directed back to the module
(03:07:36) hengsin: and with the dictionary service support, it means peopla can chose either one or the best suited.
(03:08:48) viola: i guess we run out of time
(03:09:07) viola: perhaps all should think over it and return to the point next week?
(03:09:29) CarlosRuiz: guys - I sent you an e-mail asking for comments about mercurial - please if you can take a read and comment
(03:09:38) hengsin: Jorg, yes, I think I would definitely prefer having both 1 and 3
(03:10:05) viola: aaarg ;-) I feared that ;-)
(03:10:12) hengsin: carlos, yes, agree with the move but don't like the 3 month time frame, for me, is ASAP :)
(03:10:53) hengsin: anyway, it is time for qa now
(03:11:07) viola: agree also, but would prefer at least 3 weeks
(03:11:53) trifon: what is this email about?
(03:11:59) CarlosRuiz: please send comments with copy to Tony
(03:12:05) viola: ok then bye and thanks all - also trifon and interopen!
(03:12:20) CarlosRuiz: bye - thanks
(03:12:33) interopen: bye Jorg, so time for QA?
(03:12:51) CarlosRuiz: trifon -> last message here
(03:12:51) CarlosRuiz: https://sourceforge.net/mailarchive/forum.php?forum_name=adempiere-p-m-c
(03:12:52) interopen: very interesting the Architectural topic today
(03:13:18) viola has left the channel.

Retrieved from "http://www.adempiere.com/index.php5?title=PMC_Architecture_Meeting_20100317&oldid=22909"