PMC Architecture Meeting 20100303
Date: 2010-03-03
Time: 6AM GMT
Venue: irc #adempiere-team
Support Spreadsheet: Adempiere PMC Architecture
Chat times in GMT-5
Agenda
- Review Sketch for introducing modularity
- Review Wishlist
- only priority high, per topic, starting at #8:
- author shortly explains topic
- others agree on priority
- discussion on first tasks for the topic, put them on the task list
- only priority high, per topic, starting at #8:
- Other items?
Summary
- Discussed difference between
- designing an API for a modularized core
- providing the infrastructure for modularized runtime (more important now)
- For June 14 Release: Provide a POC with demonstrates:
- sandbox, isolation, deployment and discovery for modules
- start with an "Accounting Module"
- Server-side functionality needs extra attention (EJB -> Web Services?)
- Wish #8 (and #9): Dictionary migration
- Auditing of dictionary changes
- Registration for Callouts like for Modelvalidators
- One Modelvalidator per Org
- Probably core support required (via Extension points?)
Chat
(02:00:27) ***CarlosRuiz changing cassette to architectural :-)
(02:00:38) viola: Hello all - tried to set up a simple agenda here: http://www.adempiere.com/index.php/PMC_Architecture_Meeting_20100303
(02:00:58) hengsin: hi Jorg, thx for that.
(02:01:03) viola: Any additions/changes?
(02:02:11) hengsin: not at the moment. I guess we can continue with the module topic
(02:02:24) CarlosRuiz: ok with that - thanks for the agenda Jörg
(02:02:43) CarlosRuiz: and for the document
(02:03:01) viola: ok - last time I promised to sketch how to start with modularization
(02:03:17) viola: it is here, very short: http://www.adempiere.com/index.php/RoadmapToModularization#Get_Concrete
(02:03:51) viola: main point is: We can start modularizing without waiting for OSGI
(02:04:09) viola: because the main work is: Identifying possible modules
(02:04:14) viola: and factor them out
(02:04:57) hengsin: hi Jorg, are we mixing 2 topics here - modularization of API and infrastructure for runtime module ( sandbox, isolation, deployment and discovery )
(02:05:26) viola: hmm
(02:05:41) viola: i am talking about the first
(02:06:00) viola: the latter one depends of course upon the concrete infrastructure chosen
(02:07:07) viola: which on is more important to you?
(02:07:49) hengsin: both is important but I believe it is better clearer that it should be 2 separate proposal instead of one
(02:08:39) hengsin: if I must choose one now, I would select the infra
(02:08:47) hengsin: carlos ?
(02:09:04) viola: ok then i can add another paragraph on how the service locator is implemented in OSGi and how sandbox, isolation, deployment and discovery are adressed
(02:09:48) CarlosRuiz: sorry hengsin - I'm not catching the difference
(02:10:05) CarlosRuiz: modularization of API means the implementation of interfaces?
(02:10:12) viola: right
(02:10:26) viola: and the other one: if you have an implementation,
(02:10:47) viola: how can you deploy it, and how to ensure that it does not destroy other parts of the system
(02:11:08) viola: right, hengsin?
(02:11:27) hengsin: take jdk as example, it only have an API story but doesn't have a complete module infrastructure
(02:12:00) hengsin: yes, Jorg, that's one key aspect.
(02:12:39) hengsin: so in a way, we already have an API, not very decent but we do have some but we have little module infra
(02:13:10) CarlosRuiz: an example of what you mean with module infrastructure?
(02:13:32) viola: eg DocAction
(02:13:48) viola: say, for manufacturing, you have to change the way an invoice is posted
(02:13:57) viola: DocAction is the API
(02:14:14) viola: but we can only provide a different impl by patching
(02:14:26) viola: not by deploying some module
(02:15:24) CarlosRuiz: ok, understood now, yes, I see that as long term goal - I imagine a core just acting as messaging on modules doing the work - or something like that
(02:15:32) CarlosRuiz: am I talking about the same?
(02:15:33) hengsin: carlos, right now, if you install Libero, all its classes are merge with core, a module subsystem will isolate that and minimize chances of version conflict with core ( library or classes ) and chances of breaking core.
(02:16:13) viola: carlos: yes
(02:16:28) viola: and we reach that goal by factoring more and more modules out of the core
(02:16:46) CarlosRuiz: and modules will be replaceable
(02:16:53) viola: yepp
(02:17:18) viola: my main concern is: How to convince the community
(02:17:31) hengsin: yes, replaceable, discoverable so you need infra for that
(02:17:45) viola: to provide fixes and enhancement as plugins/moduels and no longer as patches
(02:18:26) viola: eg: who will turn libero into a plugin?
(02:18:31) CarlosRuiz: I would not worry at this moment to convince community / but to ensure it can work / a POC
(02:18:53) viola: yes - my progress here is very slow
(02:19:27) viola: being stuck at that &%$/&%$ing ZK together with equinox
(02:19:31) hengsin: Jorg, is that a time constraint or there are big technical hurdle ?
(02:19:35) viola: both
(02:19:47) viola: hengsin: you proposed to help
(02:20:02) hengsin: yes, I still do :)
(02:20:14) viola: may I write you a PM and describe the setup and show you the problems?
(02:21:41) viola: but the better message is: Check out the OSGi Branch - for the swing client it works. Now.
(02:21:50) hengsin: Jorg, yes, I guess if we agree that's an important things to work on, lets continue the discussion afer this
(02:22:23) viola: Should be ok as a first POC or Carlos - what do you expect from a POC?
(02:22:49) CarlosRuiz: I would leave to you and Heng Sin to define how you prefer
(02:22:56) CarlosRuiz: if you ask me an opinion
(02:23:05) viola: I do ;-)
(02:23:20) hengsin: we have set June 14 as the next release, I guess we have at least till May 14 to produce a stable version for evaluation into release
(02:23:45) CarlosRuiz: I think we can start implementing the current interfaces in the model / and that can be the POC
(02:23:45) CarlosRuiz: and then (or in parallel) we can start thinking on the module infrastructure
(02:24:22) CarlosRuiz: I guess the easiest module to start can be accounting
(02:24:28) CarlosRuiz: it seems pretty separate from core
(02:24:30) hengsin: so lets set that as one of the target for next release for now, continue to work on branch and evaluate again for next release later
(02:26:18) hengsin: and if we can't make it for the next release, it can goes into the Oct release or the next :)
(02:26:23) viola: hengsin: ok for me (despite: remember my time constraints)
(02:27:03) hengsin: hi Jorg, as long as you are reachable on email, that's fine for me.
(02:27:23) viola: carlos: Ok - but I would need support to define the accounting interface properly - hengsin - you too?
(02:27:55) hengsin: Jorg, I will concentrate on the infra aspect first
(02:28:47) viola: hengsin: yes but it may be helpful to have a concrete example to verify approaches
(02:28:49) hengsin: maybe carlos can help on the interface definition first ( Paul will be another developer that I think can help here )
(02:28:58) viola: ok
(02:30:00) CarlosRuiz: sure, it's ok for me / mine was just an opinion, I'll work on your prefered way
(02:30:10) viola: I am setting up a new task for all PCM Arch members: POC for June 14 release
(02:30:24) CarlosRuiz: I guess I need guidance on most of this - zero experience with OSGi here
(02:31:16) viola: carlos: thats the idea: you should not need to know anything on OSGi
(02:31:33) hengsin: Jorg, do we need to replace the ejb with web service for the move to OSGi ?
(02:31:34) viola: simply define "the accounting interface" in term of java
(02:32:16) viola: ejb and osgi is a nogo
(02:32:52) viola: but - sorry for my ignorance - why are the booking done in ejbs not in pojos?
(02:33:11) viola: why not doing it the simple way?
(02:33:28) hengsin: Jorg, it puzzle most of us too, you got yo ask JJ for that :)
(02:33:40) viola: i asked JJ some time ago
(02:33:56) CarlosRuiz: what is the ejb usage on adempiere - just posting?
(02:33:59) viola: "well i wanted to learn that cool ejb stuff" - nono just joking
(02:34:01) hengsin: oh, that's interesting, wat the response then ?
(02:34:12) viola: but this is my impression
(02:34:25) viola: folks - lets get rid of that app server - no one wants it!
(02:34:52) hengsin: carlos, also to run process and report on server and ... getting database password!
(02:35:17) CarlosRuiz: ah yes, I remember
(02:35:32) hengsin: they are all implemented in 3e web services ( except the getting db password )
(02:35:52) hengsin: I guess the coming flex client should be similar to what 3e have done
(02:36:19) hengsin: the only problem with 3e is it is using an obsolete framework ( xfire )
(02:36:23) viola: hm? how can a client provide server interfaces?
(02:36:52) hengsin: Jorg, I means flex client is using web services to talk to the server
(02:36:59) viola: ah ok
(02:37:15) hengsin: for 3e, they created a delphi client
(02:37:33) viola: so if we have functionality an app server is required for
(02:38:05) viola: i think you can do remote calls to ejbs from an OSGi module or move to web services
(02:38:15) viola: only the local interface may pose problems
(02:39:00) viola: so ok maybe this topic can be closed now?
(02:39:13) viola: and we can move on with the wish list?
(02:39:29) CarlosRuiz: ok for me
(02:40:14) hengsin: yes, lets move on
(02:40:32) viola: so... #8: carlos on 2 pack?
(02:41:33) CarlosRuiz: not on 2pack specifically - it's just an option
(02:42:07) CarlosRuiz: I mean - we need to define how to manage dictionary and database for extensions
(02:42:21) viola: can we separate this in two:
(02:42:45) viola: 1. how to migrate a dict and db to a required state
(02:43:01) viola: 2. how to ensure this is done when a module is installed
(02:43:42) viola: ah so these are #8 and #9 wishes
(02:44:52) viola: for 1.: my feeling is we have community flames going around whether using 2pack or not?
(02:45:15) CarlosRuiz: yes - the issue will be similar to what we have with core classes
(02:45:15) CarlosRuiz: an extension can add tables, or columns to existing tables (easy)
(02:45:15) CarlosRuiz: but also an extension can need changes on existing entries - for example in a validation rule (hard)
(02:45:48) hengsin: I think we need to lock down on dictionary entries
(02:45:58) CarlosRuiz: viola: we don't have anything else than 2pack for this - great if we can arrive to something better, but that's what we have
(02:46:29) viola: ok - one tool is enough - we should evolve it if it isn't sufficient
(02:46:55) viola: hengsin: Lock?
(02:47:36) hengsin: I'm thinking for better audit and migration, extension/customization should not update official Dictionary entry. We should have a mechanism to allow override but not update in place
(02:48:10) viola: ok
(02:48:31) CarlosRuiz: similar as an extension must not change core classes
(02:48:36) viola: if we have modules: Why not declaring validation rules there and not in the db?
(02:48:50) viola: same for callouts, etc
(02:50:12) hengsin: Jorg, I guess in DB is for easy query and reporting
(02:50:34) viola: query and reporting on validation rules? - why?
(02:50:47) viola: I think it is for on-the-fly-changes
(02:50:50) trifon [~chatzilla@p5491A079.dip0.t-ipconnect.de] entered the channel.
(02:50:57) viola: ok you would loose that
(02:51:12) hengsin: easy to find out the active callout or validation rule - just a sql query away
(02:51:25) CarlosRuiz: holy grail :-) AD
(02:51:33) viola: hm. ok.
(02:51:34) viola: ;-)
(02:51:50) viola: I see its advantages!
(02:52:18) CarlosRuiz: but now that you touched callout - we need to implement a different callout registration way
(02:52:27) CarlosRuiz: like the change we did for ModelValidator registration
(02:52:40) viola: then back to hengsins proposal: Add module-specific val rules, callout, etc in the AD
(02:52:44) CarlosRuiz: is that architectural? to add it into the wishlist?
(02:52:46) hengsin: yes, the FR have been sitting there probably more than a year now :)
(02:53:15) viola: ah sorry how is MV regisration now?
(02:53:39) trifon: morning everyone.
(02:53:44) hengsin: Jorg, not just for the rules, but column changes like length, mandatory, etc
(02:53:45) viola: hi trifon
(02:53:54) CarlosRuiz: MV is registered now adding records to a table
(02:53:54) CarlosRuiz: hi trifon
(02:53:59) hengsin: hi Trifon, it is afternoon here :)
(02:54:09) viola: hengsin: ups thats harder
(02:54:38) viola: 2nd try: Declare callout in plugin (only there) and throw it in
(02:54:40) trifon: when we speak about modelValidation, as feature Request i had in mind ability to register modelValidator per Org.. This is needed when Org. is remote/legal entity and neeeds some specific behaviour.
(02:54:55) hengsin: right now, one big issue is when you run migration script, it can override your customization and there's no easy way to find that out.
(02:55:06) trifon: hengsin: good afternoon :)
(02:56:03) hengsin: so it will be nice if we can maintain AD changes for customization and extension as a separate record from the official dictionary entries
(02:56:25) trifon: improvement in this direction would be to store AD changes and DB changes in xml so that it is easy to review.
(02:56:26) hengsin: kind of like extend and override in Java classes
(02:57:18) viola: so add a "Module" column to every AD_XXX Table?
(02:57:18) hengsin: carlos, wdyt ?
(02:57:18) trifon: i think that here we have two problems: 1) Understand what is changed. 2) Store your customizations separwtely from the core.
(02:57:37) hengsin: trifon: talking about the 2 here
(02:57:38) trifon: but i think you talk about something differently.
(02:57:44) viola: trifon if you have 2 you dont need 1
(02:57:54) hengsin: but if you achieve 2, probably 1 is just a sql query away
(02:57:56) CarlosRuiz: hengsin: I think it must be treated like needing a change in core class - if an extension requires a change in dictionary - it means an extensibility problem
(02:58:22) viola: hengsin: What if two changes are conflicting?
(02:58:36) trifon: Carlos is right. if extension need core change then core must be improved to allow customization at this point.
(02:59:10) viola: agree but how to implement that in the AD?
(02:59:32) trifon: i think by introducing new extension points.
(02:59:39) viola: sorry guys I have to leave
(02:59:46) hengsin: exactly thats what I proposed here, an official way for people to extend/customize AD without touching official dictionary entries
(02:59:54) CarlosRuiz: the main worry IMHO will be conflict on column names - we can set up a naming strategy for extensions
(03:00:02) trifon: issue is that touching directly core is faster athen introducing extension points.
(03:00:09) CarlosRuiz: viola: thanks for attending - c u next week then
(03:00:15) hengsin: bye Jorg
(03:00:20) viola: bye all - thx
(03:00:23) trifon: bye
(03:00:26) viola has left the channel (quit: Quit: viola).
(03:00:48) trifon: Carlos, strategy in this direction is to introduce prefixes.
(03:00:52) CarlosRuiz: yep
(03:01:05) trifon: like every module to have its' own 3 letter prefix.
(03:01:46) CarlosRuiz: nobody's following at this moment (neither me) I added some columns to C_BPartner in LCO without prefix - and now I'm foreseeing problems
(03:01:50) CarlosRuiz: :-)
(03:02:10) CarlosRuiz: ok - so architectural meeting is over, right?
(03:03:07) hengsin: what if we have ad_column_ext to store customization ?
(03:03:49) trifon: hengsin: in which table?
(03:03:58) hengsin: a new table :)
(03:04:21) hengsin: ad_column_ext extend ad_column :)
(03:05:22) trifon: hengsin: you proposal is inheritance.
(03:06:32) hengsin: trifon: yes. we can discuss this later, it is qa now
