Usability - Productivity - Business - The web - Singapore & Twins

Workflow and Organisational Directory

An interesting question was raised in a recent development discussion: " What is the best way to keep the organisation data of our workflow application(s)?" My first answer was easy: "Definitely not inside the application itself". But then the choices start (too many of them are not good for you): Keep them in the Domino directory, in a custom Notes application, in a RDBMS, pull from HR, pull from LDAP? Each of these approaches do have their merit and their drawbacks. I'll elaborate.
Of course the first question to ask: Should I build my own workflow engine? There are a number of choices readily available. You can start with Ironworkflows on OpenNTF which even provides a graphical workflow editor running in the browser (and full source code included). You could have a look at XFlow, that will gain with some help based on a a great library a graphical editor in due time. It also can work with different engines. If you are not into "complete source provided" type of applications, you can checkout PAVONE Espresso Workflow which runs both on Domino and Websphere and got a nice graphical editor too. The graphical editors seem to be a must in these days, but might only determine a fraction of the usefulness of a workflow engine (I know that statement is a candidate for flame proved underwear).
But back to the initial question: Where to store the organisation data? If you build your own engine I would strongly suggest, regardless of where you store, to abstract workflow logic from storage. Define the API your application will call (like getWFDefinition(...), getApprovers(...), resolveRole(...) etc. What calls would make sense in an engine warrants a post on its own. Then code against the API (this is called contract first programming). This way it doesn't matter if your storage backend changes and you can keep the considerations for storage separate from the considerations for your applications. In Domino such an API would be best implemented using the Extensibility API (that's what XFlow is doing). Now to the storage options:
  • Domino directory (my preference)
    If there wasn't THE ADMIN the Domino directory is an excellent choice. It has ample fields to capture organisational information like department, managers etc. You even have well defined extension points (yes: places where you legally can mess with the directory).
    Big advantage: easy to sync (we give you TDI), available for other applications via LDAP and when a user goes the roles go with him. That is the general idea of a directory service.
    Biggest challenge: the admins jealously guard the directory and won't let you get anywhere near it to customise or update data. Cited reason: "If you mess with the directory, IBM won't support us" and "We can't grant the access rights without risking other parts of the data being messed up". There *is* a slight difference between altering the directory design (=fun with support) and updating the schema. Dealing with the schema (and LDAP) requires to leave the protected yellow bubble for a moment and learn about the finer details of LDAP (e.g. I didn't know that you can't use # for a group name, but & is allowed).
    Open to discussion: how well does Domino Directory serve to capture all the different roles? I've seen very few engines using the directory alone. What I have seen is the Domino directory in use for lookups, but a custom front application for data entry, so the workflow owners can't mess with other data in the directory. Also popular: sync the Domino directory using TDI with an external backend like the HR system
  • Custom Notes application
    That's the most prevalent form of corporate storage for organisation data. The original Prozessware used that approach as do the tools mentioned above. Clear advantage is the flexibility of a solution, since you can tailor it to whatever you fancy without the risks perceived in adding to the Domino directory. Of course you loose the exposure via LDAP, which makes it a solution confined to Domino (unless you add some web services, which would be a good idea) and you need to keep the Domino directory in sync with the org database. What I've seen very often are org directories that synchronise with all sorts of backends: RDBMS, HR Systems, web services etc. This of course opens the eternal question of sync vs. direct access
    As long as you look for a Domino solution there is little reason to entertain a RDBMS (unless of course your architect was hiding under a rock and thinks NoSQL isn't a database). So typically this approach can be found where e.g. HR systems run on different platforms. I'm not a big fan of direct RDBMS access since it ties you down to a rigid structure and, more important, adds another breaking point: if your RDBMS is down your workflow stops, so you need to queue your apps for processing. You can sync the data or at least use LEI/DECS to avoid language rojak
  • Pull from HR
    A good HR system would have a higher level API (probably a web service) that allows to pull org information by your workflow engine. While more business like than the RDBMS approach these APIs are rarer. If your HR system has one - good for you, use it. Again you need to decide sync vs. direct access
  • LDAP
    If your LDAP server isn't plagued with a fragile engine it is a good place to store org data in it. LDAP can be queried by a lot of platforms and is language neutral. In Domino LDAP goes well with XPages (I've seen a LDAP data source somewhere) and Java, but not so much with LotusScript. Of course if you use Domino Directory as (one of your) LDAP engine(s) you get the best of both. A good idea when working LDAP (Domino or others) is to have a sound LDAP front-end to view and enter data. Apache's Directory Studio is a good choice here
I'll cover more thoughts on what the org directory should provide in a later post. As usual YMMV.

Posted by on 21 November 2011 | Comments (4) | categories: Show-N-Tell Thursday


  1. posted by Patrick Kwinten on Tuesday 22 November 2011 AD:
    I believe that XFlow project on OpenNTF is not released yet?
  2. posted by Serdar Basegmez on Tuesday 22 November 2011 AD:
    When I saw the header, I clicked directly :)

    Because the Organization directory coming with Lotus Workflow could be a very good example how organizational data should be handled. I have worked on this for a very long time.

    It supports both hierarchical and matrix organizations and you can also put your business logic into job properties with Formula language.

    Anyone considering developing own organization repository may start from this database...
  3. posted by Thomas Bahn on Friday 10 February 2012 AD:
    For me, the only valid choice is a custom Notes application as organisational directory, if you want to use it in Notes workflow applications. Why?

    The Domino Directory only has text fields. No validation. No lists to select values from (How is the department name spelled correctly? "and" or "&" or "/" or ...? Abbreviated or long form?). No forms for teams, departments, companies, cost centers, etc. Think of renaming a department... Moving a team to another department...

    No, for me the Domino Directory is no real choice, at least not without a massive modification.

    RDBMS and LDAP: Do you really want to give up the possibility for working offline - or with a local replica to avoid "long range calls" for smaller offices?

    My way, as you pointed it out already, would be a custom Notes application, which could have some special fields and views, thus it could be even used with Directory Assistance.

    This directory would be automatically synchronized with all other systems. And because, you have the freedom to add new fields, form etc. as you like and need, it may contain all data (as necessary) from the other (HR) systems. You can detect changes in the other (perhaps leading) systems and "flag" the person's data as changed.

    If you have a new workflow application, which needs your size of shoe, just add another field.

    You can introduce functionality to "change it later" (e. g. first day of next month/quarter), as HR often nows in advance of changes, but in many systems they have to modify the data only on the day, the change happens.

    Just my 2 cent.

    Thomas Bahn
    { Link }
  4. posted by Stephan H. Wissel on Friday 10 February 2012 AD:
    @Thomas: I agree that ENTERING the data in the organisational directory needs a very custom application. However the STORAGE should be the Domino directory. There are plenty fields already and there is an extension architecture (that empty sub-forms that extend the LDAP schema). Storing all that in the Domino directory makes it not only available to Domino, but also LDAP and sync using TDI
    Emoticon biggrin.gif stw