wissel.net

Category Show-N-Tell Thursday
In a recent request by a government customer we were asked about the compliance of Domino with various RFC. I've listed them here for reference:

RFC	Description	Domino	Remarks
RFC 1123	Requirements for Internet Hosts Application and Support (STD 3)	Yes
RFC 1870	SMTP Service Extension for Message Size Declaration (obsoletes: RFC 1653)	Yes
RFC 2476	Message Submission Obsolete		see RFC 4409 below.
RFC 2505	AntiSpam Recommendations for SMTP MTAs (BCP 30)	Yes
RFC 5321	The Simple Mail Transfer Protocol (obsoletes RFC 821 aka STD 10, RFC 974, and RFC 1869, RFC 2821)	Yes
RFC 5322	Internet Message Format (obsoletes RFC 822 aka STD 11, and RFC 822)	Yes
RFC 2920	SMTP Service Extension for Command Pipelining (STD 60)	Yes
RFC 3030	SMTP Service Extensions for Transmission of Large and Binary MIME Messages	No
RFC 3207	SMTP Service Extension for Secure SMTP over Transport Layer Security (obsoletes RFC 2487)	Yes
RFC 3461	SMTP Service Extension for Delivery Status Notifications (obsoletes RFC 1891)	Yes
RFC 3462	The Multipart/Report Content Type for the Reporting of Mail System Administrative Messages (obsoletes RFC 1892)	Yes
RFC 3463	Enhanced Status Codes for SMTP (obsoletes RFC 1893 )	Yes
RFC 3464	An Extensible Message Format for Delivery Status Notifications (obsoletes RFC 1894)	Yes
RFC 3834	Recommendations for Automatic Responses to Electronic Mail	Yes
RFC 4409	Message Submission for Mail (obsoletes RFC 2476)	Partial	Domino can be configured to act as an MSA, but is not strictly compliant (e.g. does not qualify unqualified domains in addresses)
RFC 4952	Overview and Framework for Internationalized Email		This is not a standard - only an informational document
RFC 4954	SMTP Service Extension for Authentication (obsoletes RFC 2554)	Yes
RFC 5068	Email Submission Operations: Access and Accountability Requirements (BCP 134)	Yes	This is not a standard - it is a document describing a set of operational best practices. Domino can be configured to support these practices

Posted by Stephan H Wissel At 3:31 PM Location Singapore | Permalink | Comments (3) |      

Category Lotusphere2009 Show-N-Tell Thursday DXLMagic
I'll show you to take full advantage of the DXLMagic Tools one post at a time. You have exported your database application using the DesignExporter. Now you can run reports, transform the XML, edit it etc. When you are planning to work on individual element using the full DXL is a bit cumbersome, especially when you plan to write back just some parts of it later on. To split your file into multiple files you use the DesignExtractor. It will extract parts of your DXL based on an XPath expression. If XPath is still a mystery for you, this would be the time for a tutorial, Jeni's Introduction or Michael's Reference. Since you typically want to extract more than one sort of element, DesignExtractor uses a command file with instructions.
The syntax for DesignExtractor is:
java com.ibm.sg.dxlmagic.DesignExtractor [PathToDXL] [ResultPath] [CommandFile]
In the latest version of the DXLMagic.jar I have updated the path parameter, so you now can use "." for the current directory instead of specifying absolute path names. [PathToDXL] must point to a DXL File. I currently don't process all DXL files in one directory (Maybe I should think about such an option), so you have to point it to one DXL file. [ResultPath] is where your individual files go. You will see in a second, that you can specify subdirectories for individual results. The interesting part is the command file. The command file executes one extraction per line (empty lines or line with # at the beginning are ignored) and needs 3 parameters:

1. File Prefix: The extractor tries to name the file using a name or alias found in the result from the XPath expression. If it can't find something it uses a running number. It appends this to the file prefix. If your file prefix contains a directory separator it will write that file into that subdirectory (and create it if needed). So the full name for any file is composed out of [ResultPath]+[File Prefix]+[Name that the Extractor figured out]. A typical value for File Prefix would be "forms/form"
2. Export Mode: The extractor can write the result of the XPath expression into one summary file or into one file each for every result node. If you want one summary file you use "summary", for individual files use "single".
3. XPath expression: The heart of the extractor. An XPath expression is evaluated and returns a note-set. You can do whatever you fancy including using the extractor against other XML files (you might struggle with the name spaces a little bit then). To make your live easier with the XPath, the DXL namespace is abbreviated "d:". Typically you would extract high level elements like forms, views, libraries, agents etc.. However you are not limited to that. You could e.g. extract all fields to feed them into a cross-reference system. Getting started is easy: Extracting all views is this expression "/d:/database/d:view".

The command file you typically will want to use looks like this:

forms/form,single,/d:database/d:form
forms/subform,single,/d:database/d:subform
forms/page,single,/d:database/d:page
misc/frameset,single,/d:database/d:frameset
views/view,single,/d:database/d:view
views/folder,single,/d:database/d:folder
sharedstuff/field,single,/d:database/d:sharedfield
sharedstuff/action,single,/d:database/d:sharedactions
resources/image,single,/d:database/d:imageresource
resources/applet,single,/d:database/d:appletresource
code/agent,single,/d:database/d:agent
code/webservice,single,/d:database/d:webservice
code/scribtlibrary,single,/d:database/d:scriptlibrary
sharedstuff/column,single,/d:database/d:sharedcolumn
misc/outline,single,/d:database/d:outline
code/databasescript,single,/d:database/d:databasescript
misc/helpusingdocument,single,/d:database/d:helpusingdocument
misc/helpaboutdocument,single,/d:database/d:helpaboutdocument
resources/dataconnection,single,/d:database/d:dataconnection
resources/fileresource,single,/d:database/d:fileresource
resources/stylesheetresource,single,/d:database/d:stylesheetresource
resources/db2accessview,single,/d:database/d:db2accessview

Happy extracting. I'll continue that series with some of the stylesheets we have written. Stylesheets are not only good for reports, but also to whip your code into shape. Stay tuned.

Posted by Stephan H Wissel At 4:19 PM Location Singapore | Permalink | Comments (0) |      

Category Show-N-Tell Thursday
As the saying goes the complexity of any system grows at the square of the components involved. We do live in a complex world, so you can shrug, surrender to the unavoidable and move on. Or you can take a closer look and discover differences in complexity. There is the complexity from the nature of the task at hand and there is the complexity we add in the way we approach the completion of the task. Order a meal in a restaurant can be straight forward or incredibly complex. You can call that "Essential Complexity" and "Accidental Complexity", terms coined by Fred Brooks over 20 years ago. His book The Mythical Man-Month is a must read for any software engineer or normal mortal human dealing with software engineers. Brooks states that software isn't werewolfs, so there is no silver bullet. While I fully agree with his insights, I more and more doubt, that the second complexity is really "accidental". It is man made and IMHO quite deliberate. Of course we never need to assume malice where incompetence is sufficient to explain*. Let us look at the setup of your average web application (and I skipped the provisioning and monitoring pieces, as well as the diverse interfaces to administrate all this):

After careful counting that would make 6 servers or a complexity of 36. The setup is typical for Microsoft setups (Exchange, ActiveDir, Sharepoint, IIS, DotNet, SQL Server) as well as for Java setups (Apache HTTP, Tomcat/WAS, Samba, LDAP, DB2/MySQL, POP/IMAP) or your favourite scripting environment (just swap out Tomcat/WAS for the script of your choice). Not all of these servers need to be physical boxes or virtual images. They can be tasks running on one hardware. Make that high available (12 boxes) and you have a complexity of 144. Of course you could use Domino for your applications. The setup will look like this:

Of course to get there you have to challenge your believes:

• Complexity is good
• Web systems need to be 3 tier
• A directory must be dedicated LDAP (Domino does LDAP, as does AD)
• Databases must be relational
• Only complex setups scale
• Only with [insert-your-favourite-language-here] web applications can be developed properly
• And then the Lotus Domino specific ones:
  • Domino is legacy
  • Domino doesn't scale
  • NSF is not reliable
  • Web development with Domino is complicated

* On the other hand: Any sufficiently advanced incompetence is indistinguishable from malice.

Posted by Stephan H Wissel At 7:56 PM Location Singapore | Permalink | Comments (6) |      

Category Show-N-Tell Thursday
One challenge that all Notes developers, including the IBM template team, face is the customer's need or desire to customize given applications. Warren started to ask if that is a good thing. With a little planning you can design your applications so they can be easily customized without creating an upgrade nightmare (it still might be a bad dream). A very interesting example for such a "upgrade-save" extension is actually the Domino Directory (a.k.a the Name and Address book). It contains for people, groups etc. subforms that are empty. IBM will *never* touch these subforms. They are designed for schema extensions. So when you change these forms to contain additional fields, code or actions a system upgrade never will overwrite them (you need to follow some instructions to be save). In your own applications you would have these additional subforms so your customers can add functionality as needed (OK it doesn't help when you want to change the main forms).
When you want to make your behavior extensible you need to take a few extra steps:

• Move your event code from the forms and the views into a custom class.
• Ideally that custom class would inherit from a base class, that determines general behavior (e.g. validation based on a system configuration)
• Buttons, Links or Field Events can call methods of the class
• You might have the class, which obviously needs front-end methods (otherwise the On Event doesn't work) contain a class that does back-end only, so you can have the same function in your web application too.
• Now create a second/third class that inherits from the class that does all the work but leave it empty. Never touch that one, it is for your customers. That class would live in its own script library and can be overwritten by customizers without impacting your general design. Remember Interfaces live forever
• Link that class using "On event" to the form or view so the form/view events are handed down the class hierarchy until executed, see the diagram below. And no: it is not time expensive.

Posted by Stephan H Wissel At 4:51 PM Location Singapore | Permalink | Comments (3) |      

Category Show-N-Tell Thursday XPages
In our XPages workshop we have a number of exercises that deal with validation. XPages allows you 4 different approaches when and how to validate (not counting the on-field-exit validation which is a very bad habit anyway - never validate on exit, just hint, that this will not submit):

When you look at applications you typically find validation code in the submit button or attached to the submit event (that would be the second from left). While that is wide spread it is not the best way to do that in XPages. Looking at the possibilities we have 2 discussions: server vs. client, validator vs. Button/Event code. You have to be clear about the execution sequence: when a validator fails the submission code isn't executed. If the client validation (by any means) fails no server validation takes place.

• Client side validation has the clear advantage to give faster feedback, since no round-trip to the server is required. However it only can be used as a measure for user convenience since it can be canceled out using the appropriate tools (Firebug anyone) easily. Client side validation needs all information for the validation downloaded into the client. So you are limited to simple validations like required fields, consistency checks, data types etc. Anything that requires server information (e.g. Is there budget left, is this the right approver) better lives on the server side. Also currently the client side validation in XPages relies on a simple alert() to notify of validation failures and users are fast as lightning to click away prompts without reading them.
• Server side validation requires a round trip to the server (full page or a ajax call), so it will take longer. On the server side you can rely on any resources or lookups to protect data integrity. Also server side validation is less prone to manipulation (you would need access to the server sources). In XPages you also have better UI capabilities.

In conclusion: Client side is for convenience, server side for the "real stuff".

• Validation in code (a button, the submit event etc.) is a typical way validation is done. Being the prevalent way doesn't make it right <g>. You need to roll your own notification mechanism (like updating a label) and tend to tie your validation into the UI. Also when you remove a field the validation routine is likely to break. Last not least: you have a hard time documenting what gets validated and why. (You see where I'm going with that)
• Validators are defined together with a field and open a series of posibilities. XPages offers 9 different validators.
  
  You can write JavaScript, regular expressions, check for data types or roll your very own. All you can do in a button/event code you can do in a validator. Since the validators themselves don't interact with the UI the designer can decide how to surface the messages without changes to the validation code. When you remove a field all its validation code goes with it, so maintenance gets much easier. Last not least: you can run an XSLT report against your XPages source and render a report that shows a field with all the defined validators, which makes documentation easier.

In conclusion: Validators are the way to go. Read more about it in the Domino Designer WIKI on IBM DeveloperWorks.

Posted by Stephan H Wissel At 2:18 PM Location Singapore | Permalink | Comments (3) |      

Category Show-N-Tell Thursday
Reader Fields and how to handle them to balance security and performance is a never ending topic. Let us have a a closer look what actually happens. Let us assume we have a view with 500,000 document, where a particular user has access to 77 documents (which is not so uncommon in big organizations). In the first case the view is sorted by some criteria (case number, date or whatever) that scatters the 70 documents all over the database.

When the user e.g. opens that view from the web requesting the default 25 entries, the Domino server actually will more than 350,000 documents. It needs to instantiate a document object and evaluate access control rules for it. These are costly (read time and memory) operations. Performance will not meet any user expectation. Ironically this is hardly discovered by developers since they test with a few hundred documents and very often have universal access. With a little change of the view layout, the situation changes completely. We take the same view but categorize it by e.g. by @Unique(DocAuthors:DocReaders) which would list all documents for a specific reader (remember: an Author field includes Read Access Rights).

Now the Domino Server does an index search which is very fast going after a build index and reads exactly 25 documents. So with a little change in the view layout we removed 99.9929% of the document reads (bad news for hardware sellers). Looks good.... but you will say: wait a second. One user can have read access because her name is in a Reader/Author field, she might be member of a group or have a specific role, the single category only will show me one of that entries at a time.

Posted by Stephan H Wissel At 6:44 PM Location Singapore | Permalink | Comments (6) |      

Category Show-N-Tell Thursday
You are sending out emails that contain Notes links to other email systems. The router converts them to notes:// links (and with a little help even into good looking HTML mails). However when you click the document in your target database doesn't open, even if you have a Notes client installed. Now you need to investigate what is happening. These are the steps:

• Open your windows registry using regedit
• Look for HKEY_Classes_Root\Notes, which is the protocol handling definition for Lotus Notes. It should look like this:
  

  Windows Registry Editor Version 5.00
  
  [HKEY_CLASSES_ROOT\Notes]
  @="URL:Notes Protocol"
  "URL Protocol"=""
  
  [HKEY_CLASSES_ROOT\Notes\DefaultIcon]
  @="C:\\Notes\\notes.exe,0"
  
  [HKEY_CLASSES_ROOT\Notes\shell]
  
  [HKEY_CLASSES_ROOT\Notes\shell\open]
  
  [HKEY_CLASSES_ROOT\Notes\shell\open\Command]
  @="\"C:\\Notes\\notes.exe\" -defini \"%1\""
  

  Note, that your path to the Notes location might be different.
• With this fixed Notes will pickup Notes URLs and display them correctly. However there are more possible caveats. One could be the Notes URL in the email, the other the way the email application is calling the Windows function to launch Notes
• The syntax for a Notes URL is notes://servername/database/view/documentuniqueid (For a full description see the online help). If you omit the documentuniqueid Notes will open the view (the view can be specified by name or by unid), when you omit documentuniqueid and view Notes will open the database.
• When you create a link from a local database the server name is missing, so the Notes URL has a triple slash notes:///database/.... The Notes client will look for it locally. If the database is not local (or on the desktop) it will throw an error. With a little help you can force server names even if the database was created locally.
• Sometimes the mail application isn't calling the notes url properly. This is a little harder to investigate. You need to edit your registry to point the URL protocol to something else than notes.exe, so you see what is exactly handed over at the command line (You backup your registry, isn't it?). I've written a small Java class that can do that for you. So instead of @="\"C:\\Notes\\notes.exe\" -defini \"%1\"", you would write: @="\"java.exe\" C:\\temp\\CommandlineSpy.class \"%1\"". The little class will spit out a prompt with the URL handed over to the OS.

Posted by Stephan H Wissel At 12:33 PM Location Singapore | Permalink | Comments (5) |      

Category Show-N-Tell Thursday
The Domino Server has hundreds of configuration parameter and you need to be a real expert to master them all. For less smart people like the rest of us, there is the Domino Configuration Tuner (DCT). DCT can analyze R7 and R8 servers:Domino Configuration Tuner is available to any customer free of charge. It works with Lotus Domino server 7.0 and later, and runs on the Lotus Notes client, standard or basic, version 8 and later. There are no required changes to a customer domain configuration in order to take advantage of DCT evaluation.
R6.x servers are not supported, but you can run DCT and see what happens <g>
To sum it up:

1. DCT is free of charge
2. No changes are required in your Domino configuration
3. You need one R8 client to run the tool
4. You could even install a Notes client on a stick to run DCT (see this)
5. It does analyze R7 and later servers
6. There is a lot of information already available
7. The Domino wiki provides a detailed description
8. Harry Peebles posted a slidedeck on slideshare about DCT
9. You can watch a video tour about DCT
10. DCT and Domino Domain Monitoring (DDM) complement each other. So while you are on it implement both
11. DCT is available for download free of charge from the IBM Website
12. DCT runs fast, depending on your network configuration you spend just a few minutes per server

What's your excuse not to run it?

Posted by Stephan H. Wissel At 12:45 PM Location Singapore | Permalink | Comments (0) |      

Category Show-N-Tell Thursday
An interesting question arose in a discussion with a Korean customer this week. In that discussion I learned, that workflows are subject to a lot of cultural influences. Cultural influences come from the region as well as from the corporate culture. A typical requirement in workflow applications in AP: A user must not see other workflow documents (vs. the more European: doesn't want to/doesn't need to). A "must not" requirement mandates the use of reader fields (with all its performance ramifications) while a "doesn't want to/doesn't need to" requirement can be fulfilled with clever navigation (more on this in a later article). For good performance it is smart to use clever navigation in both cases.
When documents are protected by Reader Names, you typically find 3 types of entries in these reader fields: Full Names, Group Names and Roles. A Full Name (e.g. "Roger Rabit/Toons/Warner Inc" mostly stems from the users directly involved in the workflow like the requestor and the approvers, while groups and roles typically denote departments and/or business functions like warner.apps.controlling or [Audit] (I like structured group names). When a user moves on to a new department or role (e.g. from engineering to collection) (s)he would not be member of the respective groups and only will see workflow documents with direct involvement (The full name in a reader field). Depending on corporate culture that might or might not be acceptable. So what are the options to remove access for the user to these documents? What needs to be done is to make the entry in the reader field look different from the full name of that user. These options come to mind:

1. The organisation is using OU to specify the department of the users. The adminp/CA process is used to change the OU of the user moving to a new role. In this case the workflow database needs to be set not to touch names, reader or author fields. This is an advanced property in the ACL. The user name changes, the entry in the workflow database doesn't -< read access is removed.
   Caveat: Of course NO name change would be reflected in the database thereafter, so after changing your family name in marriage your workflow documents would "disappear"
2. The organisation is not using OU or the adminp for other reasons *must* update the workflow database. In this case the content of the Reader field needs to be changed. Ideally the change of role would be governed by a capable tool, so starting of the update agent can be automated. The agent would go through the workflow database and change the name of the user by prefix or appending an entry (like an OU): Roger Rabit/Toons/Warner Inc -> Roger Rabit/RoleChange/Toons/Warner Inc.
   Caveat: You need to be very clear with the auditors about such an agent. Altering a workflow document might violate corporate governance standards. It is therefore a good idea to separate the fields storing the business logic from the "access mechanic".... Which leads to the next option.
3. You need to change the application slightly. The field that computes the read access based on business logic would actually be a names field. You have another names field "DocReadersRemoved" that is computed when composed as "". The DocReaders fields would be computed and have the formula @Trim(@Unique(@Replace(DocReadersCandidate;DocReadersRemoved;""))). An update agent, similar to the previous case would only update the DocReadersRemoved field and refresh the DocReaders field. This way no business data needs to be manipulated. If that process is properly documented it can be considered "Audit save".

In any case I highly recommend to limit the number of reader and author fields per document to exactly one each: DocReaders, DocAuthors (you could use DXLMagic to do the changes). These fields most likely would be computed and pull in their values from Names fields that contain the business logic. One good tip: Always include a special role in the Author field (e.g. [Joshua] if you know). That role doesn't need to exist in the ACL, actually it shouldn't during normal operation. If the database needs to be investigated for whatever reason you add the role to the ACL and the investigator.

Posted by Stephan H Wissel At 12:40 PM Location Singapore | Permalink | Comments (3) |      

Category Show-N-Tell Thursday
A developer recently asked me how to generate dynamic questions. Dynamic as in: there is a general topic and then details. E.g.

Please state the number of building blocks per shape
Shape	Red	Blue	Yellow
Triangle	5	6	4
[Button: add row]

The filled in cells would have entry fields (not just text). So how would one do that in Domino 7 or 8 (so no XPages)? It is actually not that hard. Use a form "ConfigureQuestion" to capture all the parameters. I would have a text-area for the question (call it question) and one for the column headers (call it qdimensions). The nice trick: use a hidden regular field and construct the text-area with passthu html. The qdimensions field would be multi-value. Have a dropdown list (call it qType) with the data type. Use dijit names as alias. e.g. Text|dijit.form.TextBox. This way you will be able to generate properly validated stuff. Then have a field for width (qwidth), the first column text (qCol1) and question number (qnumber). Then have a hidden field that puts all together:
tmpCount := @Elements(qdimensions)+1;
tmpNum := "0":"1":"2":"3":"4":"5":"6":"7":"8":"9";
colCount := @Subset((tmpNum*+tmpNum);tmpCount);
tmpStart := "<table id=\"qtable"+qnumber+"\"> <tr> <th colspan=\""+tmpCount+"\" >"+question+"</th> </tr><tr>";
tmpHeader := "<th>"+qdimensions+"</th>"<;br /> tmpLineStart := "</tr><tr> <td><input type=\'text\' id=\"row1col1\" /> </td>"
tmpQuestions := "<td><input type=\"text"\ dojotype=\""+qType+"\" title=\""+qdimentions+"\" id=\"row1col"+colCount+" /> </td>";
tmpLineEnd := "</tr>"
tmpEnd := "<tr> <th colspan=\""+tmpCount+"\" ><input type=\"button\" value=\"Add Row\" onClick=\"addRow('qtable"+qnumber+"')\" / ></th> </tr>"
@Implode(tmpStart:tmpHeader:tmpLineStart:tmpQuestions:tmpLineEnd:tmpEnd;@NewLine)

The code is a rough outline, so cut and paste might not work and you also might need to work on the ids for the input fields. But you get the idea. The trick here: the multi-value field qDimensions creates as many th/td/input as we actually need. The questionnaire form would have an embedded view showing that hidden field as passthru html, so you get all questions rendered. A JavaScript function addRow(id) takes the ID of the table as parameter, so you know what table to use. Last stop: make the questions submit to an agent, so you can process all the fields.

Posted by Stephan H Wissel At 10:58 PM Location Singapore | Permalink | Comments (2) |      

Category Lotus Notes Show-N-Tell Thursday XPages
XPages feedback from an early adopter:"I'm now able to call our code from an XPage... and equivalent transactions that were taking a full second (or more) to complete in a Domino agent are now completing in 20 - 40 milliseconds. And we haven't even optimized it yet; that 50x performance boost is just from running it in XPages instead of in agents."
What do you want more?

Posted by Stephan H Wissel At 5:41 PM Location Singapore | Permalink | Comments (0) |      

Category Show-N-Tell Thursday
I'm using Linux as my base operating system. Until Domino Designer is available on Linux I need to keep a Virtual Machine with XP around. I work with both VMWare and VirtualBox. One thing you automatically do with virtualized operating systems is to shut down what is not essential. In windows the services loaded by default can be cut back quite a bit. Of course you can overdo that. One lesson I learned the hard way: The Notes 8.5 client crashes fatally if the "DCOM Server Process Launcher" isn't running. Took me 2 days of wondering until Nathan gave me that tip. Apparently that service is needed on "bare-iron" windows too.

Posted by Stephan H Wissel At 4:00 AM Location Singapore | Permalink | Comments (2) |      

Category Show-N-Tell Thursday XPages
In Domino Agents are the Swiss Army Knife of development. They can be called from the client, triggered on schedule, automatically run on an event, be associated with a web action (Open, Close) or directly called from an URL using ?OpenAgent. The ?OpenAgent use of agents is subject of this blog entry. A agent triggered from an URL can use Print statements to output results back to the browser. When the first line of your print statements is "Content-Type:..." you also can serve XML or JSON or whatever exotic format your agent logic was able to conceive. Agents come with the usual drawback: the runtime environment is intitialized new for every agent run, so you need to update a document if you want to keep track (Replication conflicts anyone?)
XPages don't have a notion of agents (you can call them in the backend, but you won't get anything back from the print statements), so it seems a step back. On closer inspection however one can see, that XPages offer a greater flexibility for the same pattern of use. Every element on an XPages has a "render" attribute which determines if the element should be rendered. "Every" includes the page itself. So when you set the page's render property to false, well the XPage engine renders nothing. So you get an empty canvas you can paint on. XPages still provides you with the page event and access to scoped variables, so you can code better performing "agents" since you can keep lookups in memory. To get the equivalent to the LotusScript print you need the ResponseWriter from the externalContext. To set the content type you use the response.setContentType also to be found in the external Context. A sample snippet you could put in the afterRenderResponse event of your page would look like this (error handling omitted):

// The external context gives access to the servlet environment
var exCon = facesContext.getExternalContext(); 

// The writer is the closest you get to a PRINT statement
// If you need to output binary data, use the stream instead
var writer = facesContext.getResponseWriter();

// The servlet's response, check the J2EE documentation what you can do
var response = exCon.getResponse();

// In this example we want to deliver xml and make sure it doesn't get cached
response.setContentType("text/xml");
response.setHeader("Cache-Control", "no-cache");

// Here all your output will be written
writer.write("<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<christmas date\"24Dec\" />");

// We tell the writer we are through
writer.endDocument();
facesContext.responseComplete();
writer.close();

More information about the FacesContext can be found in the Domino Developer Wiki.

Bonus Tip: Since you do not render anything from the XPage you just used, why not take that empty page and write the documentation for that XAgent?

Posted by Stephan H Wissel At 8:20 PM Location Singapore | Permalink | Comments (19) |      

Category Lotus Notes Show-N-Tell Thursday
My friend Lucius started his own software company Smart Toucan. His first product is Auto-User, a tool that can test Notes Classic client applications. You can use it for recording/replaying demos or to run automated software test. While IBM has a great product for web application testing (the Rational team will not talk to me anymore if I don't mention them here <g>), automated testing capabilities for the Notes classic client are painfully absent. Auto-User plugs this gap. The current Beta release works with Notes 7 and I'm looking forward to the R8 Classic and Basic configuration versions. Go check it out.

Posted by Stephan H Wissel At 1:58 AM Location Singapore | Permalink | Comments (0) |      

Category Lotus Notes Show-N-Tell Thursday SYWTBADD
Estimating software project cost and duration seems to have parallels with the local weather reports: You simply can't rely on them. Waterfall project approaches are partly to blame. Estimating the amount of work units for a requirement is another. How much work is "The system needs to be user friendly"? For the sake of this post lets us presume your have a magic box, typically labeled experience, either explicit or implicit, that gives you an insight how much code you need. Once you know how much code (a.k.a. LOC or "lines of code") you need to write, estimation becomes manageable. A good tool to estimate is COCOMOII. With the LOC, COCOMOII and your average hourly cost you can estimate the cost easily (Thanks Mitch).
However using a visual development environment (Visual Studio, Eclipse, Adobe Studio, Domino Designer) makes things a little more complicated. A form or a view you draw is not a line of code. The UML model you paint (that will turn into a database schema and code classed via generation) is not a line of code. So you need to expand your mental estimation model slightly. There are two new terms to get familiar with:

• LOC_TE = "Lines of Code - Time Equivalent": The time for painting one form is equivalent to writing how many lines of code?
• LOC_AL = "Lines of Code - Avoided Lines": When you paint one form how many lines of code can you avoid to write? The whole promise of any visual environment is, that LOC_AL > LOC_TE. This probably varies from platform to platform and makes an excellent subject for a bar brawl between competing developer camps. I guess you won't find much information about that ratio since nobody ourdays would afford to build a system twice.

Where does that leave you for Notes or Domino estimations? You need to find your magic number for LOC_TE for the various design elements (see graphic below) and then apply the COCOMOII estimation. Of course you could simply use time estimates (by revisiting your past projects), then however you need to calculate your own project variance based on project complexity and team experience. Neglecting experience and complexity are the great fallacies happily endulged in by project planners with too little experience in software project estimations. The difference for a similar sized project can be 500% from "very experienced team working on a low complexity project" to "unexperienced team working on a high complexity project". Even if you just consider experience you will still have to shell out more than double the money (230%), 70% more people and 40% more calendar time for an inexperienced team.

The orange boxes are most likely what you will get out of the discussion with the business users, including some of the yellow boxes. For the red ones you can directly insert lines of code. How do you estimate your projects?

Posted by Stephan H Wissel At 4:38 AM Location Singapore | Permalink | Comments (1) |      

Category Show-N-Tell Thursday
Keyword fields (known to the younger audience as: Radio buttons, checkboxes, combobox, dropdowns etc.) very often are populated by @Formulas. @DBColumn, @DBLookup and @GetProfileField are equally popluar. For best performance you should use @GetProfileField (if that is possible in your application context). Another way to speed things up: prevent the formulas from executing in Read Mode. If for example a formula to populate the options looks like this:

tmp := @DBLookup("Notes":"NoCache";"";"(LookupProjectsBySponsor)";@UserName;2);
@if(@IsError(tmp);"You don't sponsor projects"; tmp)

change it to

@if(!@IsDocBeingEdited;@Return(@ThisValue);"");
tmp := @DBLookup("Notes":"NoCache";"";"(LookupProjectsBySponsor)";@UserName;2);
@if(@IsError(tmp);"You don't sponsor projects"; tmp)
 

You can refine that and not populate keywords you don't intend to change (based on a status), you would update the first condition then.

Posted by Stephan H Wissel At 5:30 PM Location Singapore | Permalink | Comments (2) |      

Category Show-N-Tell Thursday
Relational data models are a very popular abstraction used in IT. And a abstraction they are rather than a mapping. I haven't come across a relational table in real live, but only documents (the ones signed by a national bank president are my personal favorites). So while document databases, objects and attributes are a better fit to the real world, RDBMS are well understood, come with a powerful query language and are reasonably standardized. Naturally you will come across the requirement to connect Notes and Domino applications to a relational back-end. The options are plenty: ODBC (bad idea), JDBC, LCLX, DECS, LEI or DB2NSF (not mentioning the 3rd party tools). Typically I see the RDBMS connections entangled in code creating more of this. A better way to separate concerns is to remove RDBMS connections from your code and let the server handle that. In Domino you can use DECS (Domino Enterprise Connection Service) and LEI (Lotus Enterprise Integrator) for that. DECS comes with Notes since some R5 version, LEI is happily sold to you by your local IBM sales rep. I will focus on DECS for this post.
The typical DECS use is to define a data connection (tip use OLEDB not ODBC to connect to MS-SQL), a data-form mapping and import the primary keys. In the data form mapping (a.k.a Activity in DECS terminology) you set what events you want to monitor: create, read, update, delete. Typically works like a charm. Any data that is updated on the RDBMS is automatically pulled into the Notes form when opened. The biggest drawback: DECS doesn't monitor record creation or deletion on the RDBMS side (that is what besides other capabilities LEI is made for). So DECS seems to be confined to cases where creation/deletion is limited to a Domino side activity. Also DECS can't trigger stored procedures. With a little creativity however you can push the use of DECS far beyond that. I'll describe some use cases I came across where DECS was used to avoid mixing Domino and RDBMS code in a function or an agent:

• Employee information is stored in a RDBMS as part of the HRMS. The employee ID is populated into the Domino Directory (yes it has a field for that). In an Notes application data is needed from the RDBMS. Instead of writing LCLSX code the application simply looks up the empPara document that is linked to the RDBMS using DECS. The document might not exist yet (if the user never had used that application before). If it does not exist it is created and populated just with the EmpID. When closed and reopened it will pull the employee information from the RDBMS. This is possible since the DECS task only monitors read/update. A scheduled agent removes documents when there is no more match in the Domino directory. In summary: if you can anticipate or know the primary key of a record you need, you can use DECS by not monitoring creation/deletion.
• Based on a workflow stored procedures in a Oracle database need to be triggered. Here some work both on the RDBMS and the Domino side was done, clearly separating both. An auxiliary table was created in Oracle with an INSERT trigger, that would execute the stored procedure using parameters given in that table and write back the success of the operation. Initially it was planned to purge the table regularly (using a Domino agent deleting the documents) but then audit loved the additional documentation, so just archival was established. During the project there were a lot of changes on both ends, however the approach of using a trigger driven table proved to be very efficient to separate the two environments minimizing interference. E.g. one stored procedure would generate a unique identifier according to some obscure, constantly changing rule. By storing that result into the aux table and creating (after reading the document linked to that table) a corresponding Notes document (again create was not monitored in that scenario) application flow was seamless.
• Enterprise parameter management [Updated] was created in a RDBMS application. Many Domino applications would need to use these parameters. Instead of having LotusScript code in every application doing RDBMS lookups DECS was used to populate and update a parameter NSF. In the activity the option "leave values in documents" was selected, so fast selections (like @DBColumn or NotesView.getColumnValues) would work. Since parameter changes happened rarely a copy of the populate keys agent from the DECSAdm database was created that would periodically shut down the activity for the parameter NSF (only the activity not the whole DECS server), pull the keys from the RDBMS based on the activity definition but only insert new keys (the original agent duplicates keys when you run it twice). Lastly it restarts the activities in DECS.

You of course can rightfully ask: why shouldn't I just use some LCLX code to connect. It isn't much more trouble. The answer is short: DECS allows separation of concern. Your Domino developers deal with what they know best: Domino. You retain the interface (and interfaces are the pieces that can make and brake an upgrade of any system) configurable outside of your own code. The whole UI for configuring and mapping has been tried and tested for years and you can open a PMR (the IBM lingo for: bug report) against it for IBM to deal with problems. You can't do that with your own code. DECS also deals with format translation and relives you from the temptation to write inefficient SQL, last not least DECS taking care of connections and connection pooling.

Posted by Stephan H Wissel At 6:16 PM Location Singapore | Permalink | Comments (7) |      

Category Show-N-Tell Thursday
In a recent meeting with a client the question was raised: "How do I support users in bandwidth challenged environments". Bandwidth challenged as in GSM (no GPRS), Modem dialup, Satellite links and the like. My first instinctive answer was replication of course. Notes was around when 9600 Baud was considered fast and replication was working then. But after reflecting on the question for a while I had to answer: It depends. You have a number of options depending on your use case. In general there are two strategies to look into: a) let data transmission happen outside the user time (a.k.a in the background) b) minimize data transmission. These are the options:

• Replication: This is the clear choice for email and informational databases like document libraries, discussions, team rooms etc. The clear advantage: while you do other things the background replication task makes sure all information reaches your desktop. It also has the clear advantage of data being available off-line. Replication is less suitable for applications where you actually only need a small subset of the whole data (typically in workflow applications) or where data transmission is very expensive. You can tweak replication settings to accommodate that. E.g. in the location "Expensive" you only receive your email (and other) database(s) while in the location Internet all is replicated (The sending of new messages is handled by the router, so it will go out). You also can limit the amount of information replicated. (See your admin help for details)
• Mail routing: In workflow applications when requesting action or approval is is usual to just send an email with a link to the workflow document. For low bandwidth situation that could be changed to send a whole form that includes the action buttons. That form could be made part of the mail design (if it is generic enough) or could be send using "Store form in document". A decision maker would get the entire information in the inbox and can click the button (which would trigger a return mail to the mail-in enabled main application. The mail is stored there as documentation and the main document is updated.
• Forms Bin: This is the "other" end of the Mail routing concept. A central database contains all the forms for all the workflow application used by bandwidth challenged users and the look-up configuration as far as possible. This database gets copied onto the workstation (either when they are in good network condition or via CD-ROM). Users fill out forms there, but the forms don't get stored in the forms bin but get emailed to a mail-in database that is the main application. You could add a non-replicating "personal bin" to keep local copies. This way only documents that are relevant to the user are transmitted. The forms bin replicates (probably receive ony), so updates to the forms, form removals or new forms are properly reflected.
• Feed enablement: To get an overview on what is happening, what action is required pulling a summary through RSS into your favorite reader. While that is a read-only approach it might fit a lot of needs. Since Domino 702 there is a feed wizzard, that can generate feeds without touching your existing application. Of course you can take a peek into IBM's and Botstation and Instant Tech provide libraries for LotusScript. Works great on mobile devices too.
• MQ Enablement: This is a variation of the Forms Bin approach. Using the Expeditor Framework in the Notes8 client you can use MQ to send the data (with a little work - sample on request) it works on R6/R7 clients too. Advantage here: you application doesn't need to worry about on-line/off-line and the data transmitted is very small. Disadvantage: you need to get used to MQ (an obviously install it)
• Web enablement: Since 4.x it has been possible to render Notes form in the browser. There is a large body of knowledge out there how to do that. Of course you want to be very light for challenged bandwidth, add compression or use XPages which does a lot of optimization for you (you want to use Firefox for its better handling of JavaScript caching)

Posted by Stephan H Wissel At 12:22 PM Location Singapore | Permalink | Comments (3) |      

Category Show-N-Tell Thursday
A lot of organizations I talk to have a lot of Notes Client applications that they want to make accessible through browsers. Some want browser only, some want dual access. All wonder how to estimate the effort needed properly. As a rule of thumb one can say: number of artifacts times time per artifact times experience of the development team. This would make one equation with three unknowns, which can't be solved. I won't discuss the "time per artifact" in this post, since this is very dependent on your technology, process and tooling used. But I will shed some light on the other two variables.
In my experience the factor skill is binary. For a guru, champion or master developer your factor is 1 (or even less), for a well experienced developer 2, for an experienced developer 4, for an entry level developer 8, for a novice 16. Of course even a novice can contribute if (s)he applies and polishes what the more experienced developers produce.
To determine the number of artifacts you would look at: forms, views, fields, columns, code events, lines of code. You easily can extract this information from a Notes database by exporting your design as DXL and then count the respective tags. Since that's a little boring let your computer do the counting. A few lines of Java will do the trick. Don't want to do that? Well, then just download this. It is sample code written in Java 6 (no Notes classes used), that will show all tag types as well as lines of code for LotusScript and @Formula. You can run it from the command line: java InspectDesign yourdatabasedesign.dxl or use it in your code.

Posted by Stephan H Wissel At 1:43 AM Location Singapore | Permalink | Comments (2) |      

Category Show-N-Tell Thursday
Every Domino server version comes with its set of system templates provided by IBM, most notably pubnames.ntf and admin4.ntf (The full list of templates is in your Administrative help file). IBM's recommendation for server upgrades is to upgrade the administration server of the Domino directory first including the templates. I recently encountered a number of customers who have concerns upgrading the address book design when they have also older server versions in their network. While it is possible to prevent replication of the design, it is more trouble than worth the effort. So let me clarify a few pointers about Domino servers and Domino system templates:

• The Domino Directory template and the other system templates are designed to be fully backward compatible. Checking the technotes you will find that we recommend to have the latest maintenance release in place before upgrading to a new main version.
• Domino Servers are designed to be forward compatible. A Domino server will read only configuration values from the Domino directory it understands. New parameters are happily ignored.
• A Domino server must run with the matching version of system templates. That it does run with older template version is lucky for you, but definitely not a supported configuration. And any of the new capabilities can't be used, since you can't configure them.
• Maintenance releases are published for a reason (Go and checkout the Fix List). IBM will not backport or provide a hotfix for problems that have been addressed by a maintenance release.

So what can you do if you are not sure what version of the templates are scattered over your Domain?
The Admin help provides the list of system templates. First you need to check if you did any customization to your version. If so, separate them as described before. Then use the admin client or a little bit of script to remove them all. You don't want to mess with different touch dates, so a bit of radical surgery is in order. Install a new server somewhere (your thumb drive is a good location) to "harvest" the original template. You don't need to generate an id or configure it. Just install and copy the NTF files from the data directory. Use the admin client to add your server and admin groups, both in their native format as well as in square brackets to the ACL (using square brackets adds these groups to all databases newly created with that templates). Copy these templates to your administrative server (pros use Replication for that). Then use the admin client to drag & drop the templates to the other servers. This will replicate them over (your adminp needs to work properly, but it does doesn't it?). You might want to do that off-hours not to get in the way of regular replication activities.

Posted by Stephan H Wissel At 11:58 AM Location Singapore | Permalink | Comments (2) |      

Category Show-N-Tell Thursday
"R8.5 and xPages ante portas!" Not before long and R8.5 will be released and you can deploy the shiny new Domino web2.0 application that created so much buzz. However you can make a safe bet, that your infrastructure team will stop you short: "Not so fast young man, we are not ready". Luckily you don't need to wait for a wholesale upgrade of the entire Domino infrastructure. You just need to add one R8.5 server:

There are a few pointers you need to be aware of:

• The R8.5 server must use the R8.5 templates for all system databases, most notably the Domino directory. R6.5x and R7.0x server work with the 8.5 system templates. If upgrading the system templates is a problem (a real or perceived one) for the infrastructure team, configure replication not to update the design from/to the R8.5 server (you do that by changing the ACLs).
• You need Domino Designer 8.5 for xPage applications. You can use Domino Designer 8.5 to design applications for R6.5/R7.0/R8.0. You just need to test them with the respective client (VMWare and friends are your friend. Or if you got a collection of PCs, use Synergy)
• You must use the Domino Administrator 8.5 to administrate a R8.5 server. You can easily administrate R6.5/R7.0/R8.0 servers using Domino Administrator 8.5
• xPages design elements don't need to be stored in the same NSF as the data. This means that you can design your web application without touching the original NSF design.
• The data sources for xPages can be on a different server. This means you don't even need to replicate your existing databases to your new R8.5 server. However you do want a fast network connection between the two servers. For the R6.5x or R7.x server the xPages application is just another set of users coming in. Of course best performance means local access using ODS48
• Never ever use a lower ODS than you can. ODS stands for On Disk Structure. Domino servers support older version of the ODS. You can use a R5 or R6/7 ODS on an R8 server. But it is not a good idea. The support is meant for the upgrade phase. You upgrade your server, the database then have the old ODS. You run compact and you are good to go (or my favorite: create a new replica). The ODS has no influence on replication. The only catch: you cant FTP a database back on a lower version of the server. This never should be an issue since skilled Domino developers don't FTP, they replicate.

Posted by Stephan H Wissel At 7:26 PM Location Singapore | Permalink | Comments (0) |      

Category Show-N-Tell Thursday
An interesting discussion happened today around Notes performance. In a rather large database (> 500k records) all documents are protected with Author and Reader fields. The access is rather narrow, so any user might see just about 1000 of the 500000 documents. Opening a view in the Notes client is rather slow and the old "rah. rah Notes is bad" song is performed. Notes performance is discussed in great length at other places, so this isn't what this post is about. I was wondering how one would implement access control on a record level in a relational database. This would be the specifications:

• Design a view that restricts access to a subset of table data (we simplify here by excluding multi-value data fields)
• A record can have zero or many readers, who are allowed to see the record
• A record can have zero or many authors, who are allowed to see the record and later update them (eventually)
• If a record has no readers any user with access to the database can see the record
• If one or more readers are present only the sum of readers and authors can see the document
• A reader or author can be of type: Person, Group, Role
• A role can be assigned to one or more Persons or Groups
• A group can contain Groups and People (we simplify here and omit the * operator)
• A group can have zero or more roles
• A person can have zero or more roles
• A person can be member in zero or more groups

Graphically it would look somehow like this:

Now how would an SQL statement look like? I'm not an SQL expert, so I might get quite some stuff wrong. But here is my go:

SELECT * FROM maintable
   WHERE maintable.id IN (SELECT readertable.maintableid FROM readertable
   WHERE readertable.entry = @CurrentUser
   OR readertable.entry IN (SELECT roletable.roleid FROM roletable WHERE roletable.entry = @CurrentUser)
   OR readertable.entry IN (SELECT grouptable.groupid FROM grouptable WHERE grouptable.entry = @CurrentUser)
   OR readertable.entry IN (SELECT roletable.roleid FROM roletable WHERE roletable.entry IN
   (SELECT grouptable.groupid FROM  grouptable WHERE grouptable.entry = @CurrentUser)
   )
   OR maintable.id NOT IN (SELECT readertable.maintableid FROM readertable)
   OR maintable.id IN (SELECT authortable.maintableid FROM authortable
   WHERE authortable.entry = @CurrentUser
   OR authortable.entry IN (SELECT roletable.roleid FROM roletable WHERE roletable.entry = @CurrentUser)
   OR authortable.entry IN (SELECT grouptable.groupid FROM grouptable WHERE grouptable.entry = @CurrentUser)
   OR authortable.entry IN (SELECT roletable.roleid FROM roletable WHERE roletable.entry IN
   (SELECT grouptable.groupid FROM  grouptable WHERE grouptable.entry = @CurrentUser)
   )

And that's without taking into account that a group could contain a group. Looks like a performance pig to me. Luckily in Domino we can use categorized views to make access fast. Of course I'm happy to learn that there are smarter SQL queries around.

Posted by Stephan H Wissel At 4:07 AM Location Singapore | Permalink | Comments (3) |      

Category XPages Show-N-Tell Thursday
I'm running the XPages enablement workshop in Beijing this week. Having to teach other about new functionality guarantees either dispair or insight. Today I had a rather insightful moment. Very often the NSF gets blasted for being a "flat-file-non-relational-whatever" file. When having a closer look at the format however you will realize, that it is a quite amazing part of technology:

• Build to function as a "whatever-you-want-to-dump-into" data store
• Build backward compatible. I still can open R2 databases in my R8.5 client
• Build to run on a client and on a server with high concurrent access
• Build to be robust: event if you tourture it (diskspace, sectors etc.) it will mostly recover
• Build to provide fulltext search access
• Build to be what you want it to be: if you happen to be a Notes client or a Domino server, it is a repository for design elements and a document database. If you happen to be the Eclipse IDE (or a webDAV client) it is a file system. If you happen to be the XPages server task it is a WAR application store. And if you happen to be David Allen, it is your trusted system for Getting-Things-Done

Posted by Stephan H Wissel At 9:03 PM Location Singapore | Permalink | Comments (0) |      

Category Show-N-Tell Thursday
When cleaning up existing applications it is good to have an overview what fields are used across forms. Unfortunately the synopsis isn't very helpful there. However with a few lines of LotusScript one can create a matrix that serves as an overview. It is a comparison by name only and doesn't tell you anything about computation mode or data-type. But it is a start. YMMV

Posted by Stephan H Wissel At 6:44 PM Location Singapore | Permalink | Comments (3) |      

Category Show-N-Tell Thursday

Letting the xPages workshop sink in for a few days, I'm now preparing to spread the love. If you happen to be in Beijing in October 15-17 2008 you could join me for the first AP xPages workshop. It will be designed as a T3 (which stands for Train-The-Trainer), so you will not only learn what xPages is all about, but also how to spread the love. I think xPages will not only be a major overhaul how existing Notes shops develop web applications, but it also offers the unique opportunity for new developers to get started on an attractive platform. What makes xPages so special?

• You only need to learn one scripting language. Both client and server run on JavaScript
• You can move code from client to server to client easily. It is after all the same language
• You have the comfort of a fast turn-around (like all scripting based web pages) combined with access to all things Java
• The database is right baked in. So no tedious xDBC configuration or scaffolding or ORM mapping. Submitted forms become documents
• Domino's security right baked in. You just declare the fields that control access and you are done (Author and Reader fields for the non-Notes readers here)
• Ajax everywhere: You can just specify that any action should refresh only a given set of elements and xPages will take care of the Ajax calls

Once you get into xPages the list will get longer. So stay tuned.

Posted by Stephan H Wissel At 9:45 PM Location Singapore | Permalink | Comments (3) |      

Category Show-N-Tell Thursday
The Dublin workshop has concluded. Others have reported and I start playing with automating upgrading your existing applications to include xPages. In a nutshell: xPages allows scripting your application end-to-end in JavaScript. JavaScript on the client and the server and just a selection for you to decide what runs where. Of course JavaScript is not a Domino server languages in older Notes versions, so you need to do something about your @Formula and LotusScript. But that's a topic for another time.
xPages are stored as XML. Existing Domino design elements can be exported with reasonably accuracy as DXL which is XML too. So with a little XSLT the both might be fit onto each other.

A Domino view starts with <view name=', while an xPage starts with <xp:viewPanel . Different names but similar structure! A column in a Notes view is represented by <column itemname=, the column in the xPage with <xp:viewColumn columnName=. Again just different names. To get from all your views to xPages follow the following steps:

• Create one xPage that looks the way your stuff should look like.
• Switch to the source view, cut and paste it into your favorite XSLT editor.
• Replace the variable parts (the columns) with XSLT template logic. Save the file into [NotesData]/xsl
• Use the DXLTools - Transformer to apply that one by one to your views (you could script that)
• Switch to the Java perspective and import the resulting files back in your NSF.
  

Posted by Stephan H Wissel At 9:06 AM Location Singapore | Permalink | Comments (0) |      

Category Show-N-Tell Thursday
A question that pops up quite regularly is how much bandwidth does Sametime need. As a rule of thumb you can use this values:

• A one to one text chat need about 0.2kBit/sec, yes that would be 200 baud. What worked in the 300Baud modem days with your favorite BBS hasn't changed: chat is cheap. Presence transfer consumes about the same (Your Netbios broadcasts probably slanders 10 times the bandwidth. If you transfer files or images (pasted into the chat) bandwidth consumption will peak as: get what you can get and finish the transfer
• Voice chat requires around 10kBit/sec per connection. A two way chat has one connection, a three way chat two (n-1 actually, you don't need network bandwidth talking to yourself)
• Video chat consumes 256kBit/sec per connection. The same rule as for voice chat applies. A threesome thus requires 512kBit/sec.

The Sametime codecs have adaptive features and you can tweak the video quality, so YMMV.
Thanks David

Posted by Stephan H Wissel At 10:31 PM Location Singapore | Permalink | Comments (3) |      

Category Show-N-Tell Thursday
When it comes to installing Lotus Notes there are a lot of conflicting ideas floating around. I'd like to shed a little light on them. Existing Notes shops typically have their Notes clients installed in C:\Notes and the data in C:\Notes\Data. This selection predates the Win95/NT/XP days. While that is working perfectly fine it makes Notes a little the odd child on the block, since all other software gets installed in C:\Program Files.
When you install Notes on a clean machine it will suggest C:\Program Files\IBM\Lotus\Notes for the program file and the data in C:\Program Files\IBM\Lotus\Notes/Data. While the first suggestion is perfectly fine I would not recommend the second. Data has no place in a program directory. Interestingly there are other options available. When you install Notes using the multi-user install Notes suggests C:\Documents and Settings\<username>\AppData\Local\Lotus\Notes\Data (on Vista C:\Documents and Settings would be C:\Users). Of course the directory names are not hard coded but pulled from the environment variables ProgramFiles and USERPROFILE. And I think that is the place where it should be in any case.
When you look at the installation on Linux, there the program goes into /opt/ibm/lotus/notes and the data in /home/<username>/lotus/notes, with shared data (the templates/help) sitting in /var/...
When you upgrade your Notes client, be it manually or using Smart-Upgrade you will realize, that the installer suggest the existing directory, thus perpetuating location decisions that might no longer be appropriate. I highly suggest to take an upgrade exercise as opportunity to rectify such decisions. How would you go about it? The best approach is to take a step back and see an upgrade exercise from a holistic point of view. You need to:

• Gather evidence that installation will work (hardware requirement, available disk space)
• Enlighten the users what's coming up/is new
• Do the actually update
• Verify the results
  One successful approach is to create a database with information around the upgrade: Users and machines, training schedules etc. Send users an invitation email that lets them know that the upgrade exercise is planned and will happen and that they are required to attend a briefing for the new features (could be a briefing using Sametime or Sametime Unyte). Make them enroll to one of the briefings by filling in a form in that database. Give enough day/time slots to get them all. The enrollment form contains code that does a preliminary workstation check: list of databases, install locations, free space etc. and records it in that database. This allows you to generate the needed scripts to clean up stuff. On the day of the briefing send another email to the user with "your session is on in xx minutes, click here to confirm your attendance". The email function would  obviously would be a button in your database. That confirmation button would then trigger the update proceedings. If you need to move the data or program files before the upgrade you need to run a little custom script that does that for you. It also would need to iron out the Registry. I personally made good experience with NSIS (given some time I'll post some samples).
  Then, while the user is attending the briefing the upgrade happens and you have the best benefit of learning: users can immediately after their training use the new functions. You need to brief long enough to give time for the installer to run. The last item the database would contain is a rating and satisfaction feedback form for the whole process.
  
  Some common problems when picking install locations for Lotus Notes:
• Installing data into the user profile and have XP sync that profile on logon/logoff. This means that you shuffle your entire Notes data on every logon/logoff since all file dates will be updated when Notes runs. Exclude the directory from sync
• Installing data into the home drive on a file server. This is a big performance hit. If you do that, at least point desktop.dsk and cache.dsk to a local drive. The rationale often is backup or the ability to work on multiple workstations. If that is a requirement you better have a look at multi-user install and the roaming user feature. This yields much better results. Storage requirements are not different, the data just moves from your file server to the Notes servers. Additional you then have access to all address books and bookmarks in the context of your Domino server and can easily run analysis and update code.
• Low hard drive maintenance. Notes databases are big compared to your average file on your workstation (having archives or replicated shared databases in the multi-gigabyte range is quite common). NTFS does a poor job (as does ext3 for that matter) in dealing with big files. They get fragmented very fast. So having a regular defragmentation running, especially before you upgrade makes a lot of sense.
  

Posted by Stephan H Wissel At 10:24 PM Location Singapore | Permalink | Comments (2) |      

Category Show-N-Tell Thursday
Disclaimer: The following considerations are hypothetical and are nothing IBM does recommend, support or implement. This are purely my little musings how something eventually could be done if someone would really want to. It also is a technical musing only and anybody acting on this musing better should study MSN's usage agreement and have a word with their lawyers.

So let us begin....
The Sametime gateway that connects a Sametime server to AIM, Yahoo or Google does currently not connect to MSN. This is not a technology shortcoming but the lack of a federation agreement between IBM and Microsoft. This post is no place to speculate who's fault that is. At the core of the problem is federation. When you use Sametime you have one identity, that makes you visible in all networks. Using a client side solution like Pidgin, Digsby, Adium, Trillian or others requires an account with each chat provider. Also it requires your firewall to be open for all these protocols for all workstations. Furthermore you won't be able to have a central logging facility which is a compliance and audit requirement in a lot of industries.
The solution approach: If Sametime traffic could run through something that handles the transition between federation and individual accounts it might do the trick (Important: this is a technical, not a legal statement!). Enter XMPP. The Sametime 7.5.1 gateway could connect to GTalk which is XMPP based. The Sametime 8.0 gateway can connect to any XMPP based server (however there is no ibm-supports-the-following-smpp-servers list). If that XMPP server connects to MSN using MSN IDs a Sametime user can talk to an MSN user. For the MSN user it will look like an MSN account (I once registered my passport account using a regular email address, so it doesn't need to be joe@msn.com or paul@hotmail.com, it can be peter.pan@disney.com). It would work somehow like that:

I spoke to Artur Hefczyc from the Tigase project. He confirmed that Tigase has a working MSN transport and that he would work with anybody to get that working. Since Tigase is an open source project, he needs to be paid for that work.

Update: Fixed the X vs. S typo.

Posted by Stephan H Wissel At 2:56 PM Location Singapore | Permalink | Comments (2) |      

Category Show-N-Tell Thursday
Steve Casteldine's blog template which is IBM's official Domino blog since Domino 7.02 got a recent boost in functionality: it now also serves as a WIKI. The Lotus team announced availability about a month ago. The following WIKIs are available:

• Lotus Quickr
• Lotus Sametime
• Lotus Connections
• Lotus Composite Applications
• Lotus Mashups

Nice side effect of choosing Domino: there is a decent Notes client UI for the WIKI and I can take it offline to places where Internet doesn't reach (yet). There is no WIKI for Notes and Domino. Given the rich material available all over the net, that wouldn't make sense (would it?). A link collection maybe, for an aggregation check PlanetLotus.

Posted by Stephan H Wissel At 5:19 PM Location Singapore | Permalink | Comments (1) |      

Category Show-N-Tell Thursday
In a recent customer session the question popped up: How can you really secure your inbox against unauthorized access including access by nosy admins. The admin part can be particularity hard since by definition administrators should have the ability to administrate which can mean to access your file. In Domino you actually can lock down the environment, even if you have to go quite a number of steps. It is a typical case of "who controls the controller" This is my wash-list:

Posted by Stephan H Wissel At 1:26 PM Location Singapore | Permalink | Comments (6) |      

Category Show-N-Tell Thursday
The good news: most of the skills that will help you to excel in Domino are generic and can be applied to any development environment.
The bad news: there is a lot of stuff to learn. I'm compiling a roadmap for (Domino) Developers wannabees taking a little broader approach. This is my first draft:

In the coming days and weeks I will discuss/fill each of this circles with details and recommended readings/training material. Feedback is highly appreciated.

Posted by Stephan H Wissel At 9:14 PM Location Singapore | Permalink | Comments (8) |      

Category Show-N-Tell Thursday
It is all out there. Sometimes it is hard to find. Support has compiled various lists with "most wanted documents":

• Notes Client
• Lotus Domino Server
• Domino Web Access (DWA, also known as iNotes)
• Notes & Domino FAQ
• Notes & Domino Best Practices
• Blackberry & Domino 8
• AdminP FAQ
• Sametime FAQ
• Easy Sync Pro

Enjoy!

Bonus Track: The official history of Lotus Notes and the wiki version

Posted by Stephan H Wissel At 12:03 PM Location Singapore | Permalink | Comments (0) |      

Category Blog Show-N-Tell Thursday
I had an iteresting discussion with a customer this week. They use Domino and dotNet for their web applications. Their decission criteria when to use what: if the data of the application needs to be fed into their data warehouse at a later point of time, they use dotNet since storage there typically ends up in an RDBMS. The biggest problem they face, in their own voice: "Our users are pretty spoiled from Domino. They expect days as turnaround time for applications. Using dotNet it takes at least three times longer."
So I asked why they don't use DECS to connect to the RDBMS. They could develop the application in Notes/Domino and once the app does what the user wants just add the tables in the RDBMS and link them up using DECS. They asked back if there is a way to generate the table or at least the create table statement from Domino directly. The short answer: Yes, you can, however you need to make decissions on datatypes and field length. The long answer: you need Domino Designer (for the Tools - DXL Utilities - Transformer ... menu) and a little XSLT stylesheet.

Posted by Stephan H Wissel At 11:37 AM Location Singapore | Permalink | Comments (7) |      

Category Show-N-Tell Thursday
Application performance in Notes and Domino can greatly vary depending on the number of views and the view selection formulas you use. When inheriting databases applications for maintainance there is no real easy way to get an overview what view selection formulas have been used. So I did write myself a function that creates a document with such an overview table.See the function below. To test it I simply copy it into an agent and call it for the current database.
Of course you could think of running it against multiple databases or altering the html with some Ajax stuff to make it sortable. Here is my test agent:

Option Public
Option Declare

Sub Initialize
	Dim s As New NotesSession
	Dim db As NotesDatabase
	Dim doc As NotesDocument
	
	Set db = s.CurrentDatabase
	Set doc = db.CreateDocument
	Call ReportViewSelectionFormulas(db, doc)
	Call doc.send(False,s.UserName)
	
End Sub

Update: Sorry folks, got the wrong code in the core functionsm fixed now. Enjoy!

Posted by Stephan H Wissel At 2:39 AM Location Singapore | Permalink | Comments (5) |      

Category Show-N-Tell Thursday
In Part 1 I introduced the approach to code injection and some sample LotusScript Code to pull a design element, transform it and push it back. Missing there was the function that actually implemented that method. I'm using a Java class to do that, which I wrap into LS2J. The LotusScript function looks like this:

Option Public
Option Declare
Uselsx "*javacon"
Use "CodeInjector"
Function injectCode(rawDXL As String, xPath As String, codeSnippet As String) As String
	'Dummy doesn't do anything
	Dim jSession As JAVASESSION
	Dim injectorClass As JAVACLASS
	Dim injector As JavaObject
	Set jSession = New JAVASESSION
	injectCode = rawDXL
	
	Set injectorClass = jSession.GetClass("DominoXPathInjector")
	Set injector = injectorClass.CreateObject
	
	'Now set the document
	Call injector.setDocument(rawDXL)
	'and execute the function	
	injectCode = injector.injectCode(codeSnippet, xPath, 4)
	
End Function

Now let us have a quick look at the Java class.

Posted by Stephan H Wissel At 11:08 PM Location Singapore | Permalink | Comments (0) |      

Category Show-N-Tell Thursday
In an earlier entry I discussed how to maintain custom system databases. It works well when you add design elements or change them wholesale. However it doesn't work very well when you tweak design elements. Something like: adding one action in a form or using a different subform for a specific task. Once an updated system template is available, you have to go back and recreate all your changes in your template.
This entry discusses how to automate these little fixes. There are a number of moving parts involved:

1. Export the design element using DXL. (You might need to set NotesDXLExporter.RichTextOption = RICHTEXTOPTION_RAW to make sure the fidelity of forms is maintained)
2. Locate the entry/replace point. The best mechanism for that is XPath
3. Locate the new code. This could be as easy as provide a string with the XML Snippet to pointing to an existing design element elsewhere. (We would use XPath again)
4. Perform the insertion/replacement. We will have the option: before/after/replace
5. Write back your changes and test it

What type of code injections come to my mind:

• Add an action to a form/view
• Add a subform to a form
• Replace a validation formula in a field
• Insert a script library and a custom class into a form
• Add a field at the top or bottom of a form (our favorite hidden fields $$Return anyone

Understanding DXL and XPath is key to make this work. As a Domino developer (and you are one? Otherwise all this might sound Greek and Latin to you) you have seen DXL and its description in the help file. XPath might be less familiar. In a nutshell: XPath is to XML what SQL is to relational data. When you develop XML Stylesheets (a.k.a. XSLT) you use XPath selectors to define your template matches. XPath is a set language, so you better remember your kindergarden days (give me all shapes that are blue, triangle and big). It is easy to get an overview and to get started and can get as complex as you can take it. One little caveat (you see Latin is creeping it): XPath is case sensitive
Some simple examples:

• * selects ANY element.
• / selects the root element, basically before any tags in the XML
• /database/form/actionbar selects the actionbar elements in all forms in our XML
• /database/form[1]/actionbar selects the actionbar element in the first form in our XML
• /database/form[@name='Message']/actionbar selects the actionbar element in the form Message in our XML
• /database/form[@name='Message'] //field[@name='Subject'] /code[@event='defaultvalue'] /formula Select the default value formula for the field "Subject". Please note the double slashes in front of the field. They are intentional. One slash says: a child, two slashes says any decendant down the document. Also: no spaces before/after the slashes. They are here for readablitity only.
• //field[code/@event='inputvalidation'] Any field that has an input validation formula

You get the idea. You want to have a good XML editor.

Posted by Stephan H Wissel At 12:46 PM Location Singapore | Permalink | Comments (2) |      

Category IBM Show-N-Tell Thursday
From the "Your-Notes-client-is-different-department".

When replying to an email Notes does some address shuffling for you. As long as you have one sender and one recipient, it is quite logical. The original sender become the recipient, the recipient becomes the sender. Everybody in the cc list stays in the cc list.
 to  

However that doesn't work anymore when you had more than one recipient in the To field. The general rule (as I once understood it) for when to place in To and when to place in CC is:
- To: This is an actionable item for the receipient
- CC: for information only. (In this days it is more like a "Just in Case Copy)

So what should your email system do, when there is more than one person in the TO list?

a) Presume that your reply to the original sender is actionable for the other recipients too?
b) Presume that your reply to the original sender is for information only for the other recipients?

Notes uses presumption b) while other email systems use presumption a). So you go from:
 to

Posted by Stephan H Wissel At 11:10 AM Location Singapore | Permalink | Comments (0) |      

Category Show-N-Tell Thursday
Rocky posted a great idea on IdeaJam: Make DXL complete and roundtrip save. I'm a big fan of DXL and really really want that feature. So go over to IdeaJam and cast your vote!

Posted by Stephan H Wissel At 10:48 AM Location Singapore | Permalink | Comments (1) |      

Category Lotusphere2008 Show-N-Tell Thursday
Based on the feedback on my little download script I'd like to share how I created it.
Step 1: Have a look at the database that hosts the downloads:
Step 2: Pick a view
Step 3: The same view as XML and all of it.

Posted by Stephan H Wissel At 12:08 AM Location Singapore | Permalink | Comments (1) |      

Category Show-N-Tell Thursday
The R8 inbox, calendar and contacts take full advantage of the new Java views. However support for your application has been postponed to the NMFR*. Memento bene: "support for". That doesn't mean that you can't do it. After all the mailbox is just a Notes application. The biggest challenge: in the normal Designer there seems to be no way to specify the additional properties for the Java views like side preview or the business card view. Actually there is. In the R8 reviewers guide on page 155 (that would be PDF page 163) a new notes.ini parameter is documented:

Designer_ShowPropForJavaViewsUI=1

Add this parameter to your local notes.ini and designer will reveal new properties in views (the (i) tab in the view and the propeller head tab for a view column). The redpaper also mentions what you can do with this. One interesting piece: Currently we have a viewer type: "Table" and one "Tiled". You can specify both and you will get a switcher (like in contacts). The API will be documented, so you could render your very own views (Pivot anybody?). However if you add all this properties nothing happens. A few pieces are missing (and since it is not automatic, there's no support yet). Again the redpaper helps: on page 146 (PDF 158) there is a note, that reminds you that you can link a frameset to a composite. So what you need to do for your JavaView enabled views is to use a composite application. That is actually nice, since you have full backward compatibility: Prior to R8 the frameset opens. In R8 the composite opens.

Posted by Stephan H. Wissel At 8:00 AM Location Singapore | Permalink | Comments (4) |      

Category Show-N-Tell Thursday
Mental Note to self:

If you write something like:

NotesStream.write(someStringInside)

the compiler will not warn you, that you shall not provide String where Byte is needed. But the runtime will take care of you! It will terminate the execution of your subroutine (not the entire script, just the subroutine) without error message, because you shall not write String where Byte is needed and before you commit this blunder again the routine terminates. And don't get me started on Streams, Transformers and character sets.. With this little hickups Blizz is progressing.

Posted by Stephan H Wissel At 5:03 AM Location Singapore | Permalink | Comments (0) |      

Category Show-N-Tell Thursday
In the beginning the world was easy. You wanted to notify a user, just use @MailSend with [IncludeDocLink] and your email notification was done. It was obvious if your application was on a Notes client, your messaging application would be Notes as well.
Today the picture looks different. You might access your email through a portlet on the intranet or through a POP3 client. But you still want to be able to open that link in the Notes client right? I'm not aiming at web enablement here. What are your options:

• In a Notes client you can use NotesRichText.AppendDocLink
• You could send a link with notes:// ...
• You can create an attachment as NDL file: To see what you need in there you copy a document link and paste it into Notepad. You will get this result:
  
  Wissel's Address Book - My Contacts
  <NDL>
  <REPLICA 4825713F:00352EEC>
  <VIEW OF85255E01:001356A8-ON852554C2:00753106>
  <NOTE OFE3DB4DB9:F46B6DAE-ON482572F9:00366DC6>
  <HINT>CN=d23m0120/OU=23/OU=M/O=IBM</HINT>
  <REM>Wissel's Address Book</REM>
  </NDL>
  
  Looks utterly familiar. Inside the NOTE the document uniqueID is embedded similar to the value in the property box. The stuff around the NotesDocument.UniversalID is static, so we can construct that. The view is the uniqueID of the design document. Luckily you can replace that with the name or alias of that view

Since all 3 cases are needed over and over again I added a little utility class to my script tools.

Posted by Stephan H Wissel At 1:00 AM Location | Permalink | Comments (4) |      

Category Show-N-Tell Thursday
In a recent training session a participant asked why Notes forms are so ugly. Naturally I pointed him to Chris. Nevertheless he got me thinking. How many steps does it take to make a Notes form looking nice. Let us start with a very typical basic form:



The first thing will be the background for toolbar and form. I borrow some style from Lotus Connections: adding background images leads here:



Next step is to take care of the radio buttons, mandatory fields and the borders.

Posted by Stephan H Wissel At 1:00 AM Location | Permalink | Comments (7) |      

Category Show-N-Tell Thursday
This is an IBM announcement our marketing manager asked me (very nicely) to publish here.

---

The recent release of the IBM Lotus Notes and Domino 8 platform gives developers more integration and business solution options than ever before. To celebrate this extensibility we would like to offer our Business Partners free access to Lotus development specialists who will assist you in creating Notes and Domino 8 business solutions.

Your existing solutions will continue to run in Lotus Notes and Domino 8 without modification -- all new integration options supplement existing application development capabilities. Our Lotus development specialists will work with you to explore possible integration scenarios and guide you through the details of this new set of integration options including:

The Composite Application Framework
The Lotus Notes 8 platform allows developers to build reusable, loosely coupled components which can be assembled into composite applications. These reusable components can be created from NSF, Eclipse or IBM Lotus Expeditor technology. Our specialists can evaluate your requirements and propose strategies for developing reusable components or improving integration with a composite application.

The Sidebar and Toolbar
The Lotus Notes 8 sidebar and toolbar are in the end user's sight at all times so they provide an excellent opportunity to display a ubiquitous view of your application or service. Our specialists can show you how to integrate components of your solution into the sidebar and toolbar most effectively. Work you've done developing for Lotus Sametime Connect will pay dividends here as well -- most plug-ins developed for IBM Lotus Sametime 7.5.1 are compatible with Lotus Notes 8.

Web Services
Lotus Notes and Domino 8 provide comprehensive support for web services producers and consumers on the Notes client and Domino server. Our specialists show you how to incorporate your web services in Notes and Domino applications to capitalise on existing IT investments and realise a service-oriented architecture.

Lotus Domino and DB2 Integration
Lotus Domino 8 provides a fresh opportunity for Business Partners with relational database experience and solutions. With Lotus Domino and DB2 integration developers can expose NSF data for relational use and list DBMS data in views and embedded views. Our specialists can discuss the possibilities afforded by this integration such as report generation and integration with other relational data stores.


To learn more about all that Lotus Notes and Domino 8 has to offer, please have a look at "Building on the evolution of people-centric applications" or "Creating business flexibility with IBM Lotus application development software" on the Lotus Notes and Domino developer site. For more information on composite applications see the composite applications page on developerWorks, or the composite applications blog.

If you'd like to find out more about how we can help you take advantage of Lotus Notes and Domino 8 please email us at ndiscool@us.ibm.com. We'll coordinate a follow-on call to review your needs and get you started right away.

At Lotusphere 2008 we will be looking to highlight Business Partner solutions that exploit the new integration possibilities in Lotus Notes and Domino 8 - including a new award category for these solutions. We encourage you to take advantage of this offer while it's available in preparation for Lotusphere.


Thank You,

Tim Shortley
Program Director, Worldwide ISV Technical Enablement, IBM Lotus Software

Posted by Stephan H Wissel At 10:36 AM Location | Permalink | Comments (4) |      

Category Show-N-Tell Thursday
I tried to draw a generic RDBMS schema description for a note. This is my first draft:

Posted by Stephan H Wissel At 12:03 PM Location | Permalink | Comments (3) |      

Category Show-N-Tell Thursday
When you upgrade a server there is always the possibility, that something goes wrong (you did a full backup,did you?) when running the new version. So you need to have a "Plan B" how to fall back to the original version. When upgrading from Domino 6.x to Domino 7.x or 8.x you can prepare a small script that allows to automate the fall-back and the fall-forward. This is what you need to do:

Let us presume your Domino program files location is D:\Domino and your data location is E:\Domino\data

1. Shut down your Domino server
2. Copy the program directory:

   COPY D:\Domino\*.* D:\Domino6\*.* /S /E /V

3. ZIP away all system templates in E:\Domino\data\*.ntf (We will restore the ones that are not provided in the update
4. Install Domino8 as update into D:\Domino and E:\Domino\Data
5. Unzip the templates from Step 3. Do not overwrite exisiting files. Older versions of Domino run fine with the new system templates but the new Domino servers must have the new templates.
6. Create the file SwitchDomino.cmd in a convenient location (Desktop?) with the following content:
   

   @ECHO OFF
   CLS
   NET STOP  "Lotus Domino Server (Dominodata)"   <<<---- You need to check the exact name of your service 
   IF EXIST D:\Domino6\*.* GOTO R8TO6
   IF EXIST D:\Domino8\*.* GOTO R6TO8
   ECHO Something is wrong - NEITHER D:\Domino8 nor D:\Domino6 seems to exist
   PAUSE
   GOTO EXIT
   :R8TO6
   RENAME D:\Domino Domino8
   RENAME D:\Domino6 Domino
   GOTO EXIT
   :R6TO8
   RENAME D:\Domino Domino6
   RENAME D:\Domino8 Domino
   GOTO EXIT
   :EXIT
   NET START  "Lotus Domino Server (Dominodata)"
   

7. Restart your server

If you need to fall back you just execute the cmd and after a few minutes you switched versions. Caveat for R8: R8 can use the ODS48 format. Once you are there, there is no fast way back since you would need to use compact to revert to the previous ODS format which can take quite some time.
Remark: You do data backup do you?

On Linux systems you play with symlinks, but that's a story for a different time.

Posted by Stephan H Wissel At 4:27 PM Location | Permalink | Comments (0) |      

Category Show-N-Tell Thursday
In a recent discussion a customer asked how much space could they save on their mail servers if they move all attachments to Lotus Quickr or an content management solution like IBM DB/2 CommonStore for Domino. While you can inspect a database easily for size and number of documents it doesn't tell you how much of this size is attributed to attachments. Since altering the mail template to add an attachment view was out of the picture I whipped up a little script, that will scan an entire server and report back the size figures into a database:

'ScanForSize: 

Option Public
Option Declare
Use "SizeTools"
Sub Initialize
	Dim s As New NotesSession
	Dim db As NotesDatabase
	Dim curDB As NotesDatabase
	Dim dbDir As NotesDbDirectory
	Dim serverName As String
	
	Set db = s.CurrentDatabase
	If db.Server = "" Then
		serverName = "(local)"
	Else 
		serverName = db.Server
	End If
	serverName = Inputbox("Please select the server to scan (local) for local","Server Scan",serverName)
	If serverName = "" Then
		Exit Sub
	End If
	
	If servername = "(local)" Then
		Set dbDir = New NotesDbDirectory("")
	Else
		Set dbDir = New NotesDbDirectory(serverName)
	End If
	
	Set curDB = dbdir.GetFirstDatabase(TEMPLATE_CANDIDATE)
	Do Until curDB Is Nothing
		Print "Working on " & curDB.FilePath
		Call countThis(curDB, db)
		Set curDB = dbDir.GetNextDatabase
	Loop
	Msgbox "Run completed"
End Sub


Sub countThis(curDB As NotesDatabase, reportDB As NotesDatabase)
	Dim curSize As DBSizeInfo	
	On Error Resume Next
	Set curSize = New DBSizeInfo(curDB)
	Call curSize.count
	'We save only database information that has an attachment
	If curSize.AttachmentCount > 0 Then
		Call curSize.save(reportDB)
	End If
End Sub

Posted by Stephan H Wissel At 1:00 AM Location | Permalink | Comments (7) |      

Category Show-N-Tell Thursday
My colleagues Pong Yawagun and Nithitus Upatumvipanon from IBM Thailand have started their Lotus Notes Blog. They will provide their insights into Lotus technologies and also translate articles from others (including this blog) into Thai language. Go and welcome them! And yes NotesSensei seems to go into franchise.

Posted by Stephan H Wissel At 11:08 AM Location | Permalink | Comments (0) |      

Category Show-N-Tell Thursday
Domino upgrades are comparable easy and can be quite fast. Of course you need to know what you are doing. There might be times when you don't want to have a smooth ride. Like when you are paid by the hour or when you want to move to a different platform. In this case this post is for you. Follow the following easy to implement steps and your upgrade will be an utter failure. But be aware, you have to set the stage before the upgrade, voicing concerns, so you can blame it on IBM when things start to fail --- and they will.

1. Do not upgrade the admin's workstation. Using the latest version of Domino administrator would actually give you access to the latest feature, avoid this
2. Make sure, that you don't have access to all servers and that you are not a system administrator of the domain
3. Make sure, that you can't reach all the servers from the admin workstation
4. Start upgrading the most critical application server first, and the hubs and the administrative server last
5. Make sure you have enough time so the design task on an old server can undo all design changes the upgrade process might have applied
6. Make sure you have touched and customized all system templates, so their change date is younger than the new version. Also make sure the templates don't replicate everywhere. A mess in the system template will give you weeks of (billable) fun
7. Make sure, that your system databases don't replicate or replicate outside the deletion stub life cycle interval
8. Never ever upgrade the design of the Domino Directory or other system databases. If you do so the upgraded server actually could use other settings than defaults.
9. Try to make sure that your ODS is at least one version behind. This keeps the indexer busy.
10. Yes a full hard disk is also a good idea: if the available space is less than your biggest database and less than 20% that is good
11. Enable disk compression. Having the OS scream when it needs to uncompress notes databases on access
12. Use a desktop virus scanner on your server. Make sure it scans every file on access
13. Don't uninstall any 3rd party extensions (like the virus scanner or compliance plug-ins). It is more fun to search for such issues. And while you are on it: don't use the latest version of 3rd party apps, they actually might be compatible with the new version of Domino.
14. Enable transaction logging with either the OS or the data drive as destination. Hard disks should spin hard.
15. Make sure you don't apply your OS level service packs or patches. This is especially important on W2K3 and W2K3/64.
16. Do not apply the latest version of your source version. (LIke 5.0.13a when you come from R5, or 6.5.6 when you come from R6.x) This is especially fun when skipping a version
17. Make sure that the messaging people don't talk to the apps people vice versa. Also don't give notice to the apps people that you do the upgrade
18. Do not maintain documentation which applications use which system template for lookup. Anyway documentation is for whimps.
19. Be creative. Look for best practices in Redbooks, on the IBM site and other online locations. Put a "not" or "don't" in front of every recommendation.
20. Don't use fixup or compact before or after the upgrade.

This list is by far not complete. So if you know another way to screw your upgrade, let me know.

Posted by Stephan H Wissel At 12:07 PM Location | Permalink | Comments (3) |      

Category Show-N-Tell Thursday
Everything in Notes can be customized by tweaking with the design. I highly recommend never to tweak design in a database itself, but use Domino's template capability. You can have multiple databases using the same layout. Even if you build a specialized custom application you will have at least 3 databases sharing the same design: development sample data, UAT sample data and production data. So use template, and NO hiding the design is not a nice thing to do and actually not a security feature (it's rather an obscurity feature).
In a lot of installations I came across system databases have been altered: the Domino directory, the log, the mail template etc. IBM's support always has a hard time helping in this environments. There is a rather strict policy to insist on original IBM templates. Following a few easy steps however you can eat your cake and have it too. The secret is to use the template system wisely. Following the steps has quite some advantages:

• Your customizations are protected from being overwritten since they are sitting in their own database
• Your customizations are visible since they are sitting in their own database
• Since the original templates are untouched future fixes are easily included in your actual database
• You can switch back and forth between the IBM template and your customized template using "replace design". This is extremely useful when troubleshooting issues: switch to the original IBM template, if the problem is gone you know where to look. If the problem persists you don't need to argue with support about customization having an impact

These are the steps:

Posted by Stephan H Wissel At 2:07 PM Location | Permalink | Comments (3) |      

Category Show-N-Tell Thursday
I got quite some requests on the details of the syncing documents in views pattern/anti-pattern.So this is what you need to do:

1. Have 2 views with identical keys, sorted by the key
2. Julian's OpenLog database
3. The main function SyncAllDocument
4. 3 auxiliary functions: getCompareKeySource, getCompareKeyTarget and updateSourceDoc

In my example code I made the assumption, that the documents contain the same fields and have a single key. You might have different requirements like computing things and/or only coying part of the items or do a computewithform before saving. Nevertheless you don't need to touch the main routine for that. You get the idea.

Posted by Stephan H Wissel At 9:13 PM Location | Permalink | Comments (3) |      

Category Show-N-Tell Thursday
When reviewing agents or trouble shooting mis-behaving databases I come across a popular anti-pattern for Domino: delete and recreate. Anti-pattern are well known failures for common known problems. "Delete and Recreate" can be found in report generating applications or databases that pull values from other sources like RDBMS or flat files. Mostly you can find that pattern when developers came from a RDMBS background where deleting equals the total removal of information. In Domino things are a little different. When you delete a document something remains back: a deletion stub. A deletion stub is the DocumentUniqueId plus a flag that says "I'm a deletion stub". The stub is then replicated to other servers or clients to also remove this document in the replica. The default life span of a deletion stub is 30 days.
In a recent analysis I was looking at a database with just 400 documents, that was big, slow and prone to crashes. On closer examination I found an agent performing this pattern on the 400 documents every 30 minutes. Do your math: 400 x 48 x 30 = 576000. Quite some baggage for just 400 entries.
So what are the alternatives?
Requirements like this are often solved with another anti-pattern: "Loop through a big loop and do a dbLookup (or getDocumentByKey) for each iteration". But there is an easier way: Typically there is a sorted key (customer number, part number, record id etc.) that would be available in source and target. Create a collection for source and target and use what I call "The Tango" or "The Wiggle". The pseudo code looks like this:

1. Read first source key and first target key
2. Do until you run out of keys
3.  if source key is equal target key: call sub routine comparing the two records how they have to be compared and update and save target if it has changed (but not otherwise)
4. if source key is bigger than target key: get next target key and delete current target
5. if target key is bigger than source key: insert source document into target view and read next source document
6. End of the do loop

Of course your production code needs to handle the case that you run out of source keys (= delete all remaining target documents) or target keys (=insert all remaining source documents).
This way you don't create unnecessary zombie deletion stubs.

Posted by Stephan H Wissel At 11:05 PM Location | Permalink | Comments (4) |      

Category Show-N-Tell Thursday
In Part 1 of this Show-N-Tell Thursday I showed how to generate a WSDL definition out of a Notes form. While this is nice, it leaves quite a bit of work to actually implement the service. In this installment we take the opposite route. Instead of generating a WSDL definition, LotusScript code will be generated out of a Form definition. Unfortunately DXL doesn't support Web services very well (at least up to R8.0), so I will generate a text file that contains pure LotusScript code. This text file needs to be manually imported into an empty Web service (Copy and paste will do).

I generate a few classes including a custom class which you can overwrite to implement your specific additions to the code. You might want to put that into a different library, so you keep your customization. Nota bene: the generated code is not real production quality. I haven't implemented good logging or error handling (OpenLog anybody). Once you see the code you get the idea and can complete that easily (feedback would be nice).

A web service requires a LotusScript class with public members, properties and/or functions. Domino Designer will then generate a WSDL file for us. One path to travel would be implementing generic code, that takes fields as name/value pairs. While tempting this has the clear disadvantage, that the web service becomes rather unspecific and we can't take advantage of a strong validation using XML schema. So my approach is to analyze the form and generate 2 XML structures: one for all input fields, used to create and update a document and one for all fields used to read a document. I don't take hide-when formulas into account.

There are a few challenges to overcome. First we need to deal with multi-value fields. In WSDL these are expressed as Arrays. Secondly we need to map the Domino data types to WSDL data types. Luckily half of the work is done since Domino includes "lsxds.lss" It translates WSDL data types into LotusScript. From LotusScript to NotesItems is a path well travelled.

Let's have a look at the XSLT stylesheet (if you don't know how to translate DXL using XSLT use this plug-in -- you are using FireFox are you?)

To instruct the XSLT processor to output plain text rather than XML or HTML we use this statement:
<xsl:output indent="yes" method="text" omit-xml-declaration="yes" media-type="text/lotuscript" xml:space="preserve" />

Then inside the root tag the LotusScript Code is written as you would write in Domino Designer.
  <xsl:template match="/">
        <!-- We use the build-in data type mapping for LotusScript agents -->
        %INCLUDE "lsxsd.lss"
        '  Autogenerated code to be embedded into a Domino Web Service
       
        Public Class form<xsl:value-of select="$formName"/>

We have 2 levels of comments here. XSLT Comments looking like this: <!-- ... --> and regular LotusScript comments starting with '.
Whenever we need to get specific we can use <xsl:value-of to pick a value from the DXL form.

The class we will expose into the webservice is, other than its name and the content of the parameters pretty standard and (IMHO) self explaining:

Public Class form<xsl:value-of select="$formName"/>
           
            '/**
            '* createData takes in the custom form data and creates a new document.
            '* the data given as parameter matches all input fields of the form
            '*
            '**/
            Public Function createData(formData as <xsl:value-of select="$formName"/>InputData) as ResultCode
                    Dim curData as new <xsl:value-of select="$formName"/>CustomData
                    Set createData = curData.createData(formData)
            End Function

        Public Function createAndReadData(formData as <xsl:value-of select="$formName"/>InputData) as ExtendedResult
                Set createAndReadData = new ExtendedResult
                Set createAndReadData.Data = new <xsl:value-of select="$formName"/>CustomData
                Set createAndReadData.Result = createAndReadData.Data.createData(formData)
        End Function

        Public Function readData(key as String) as <xsl:value-of select="$formName"/>CustomData
                    Dim tmpResult as new <xsl:value-of select="$formName"/>CustomData
                    Call tmpResult.readData(key)
                    Set readData = tmpResult
            End Function
           
            Public Function updateData(key as String, formData as <xsl:value-of select="$formName"/>InputData) as ResultCode
                    Dim curData as new <xsl:value-of select="$formName"/>CustomData
                    Set updateData = curData.updateData(key, formData)        
            End Function
           
            Public Function updateAndReadData(key as String, formData as <xsl:value-of select="$formName"/>InputData) as ExtendedResult
                    Set updateAndReadData = new ExtendedResult
                    Set updateAndReadData.Data = new <xsl:value-of select="$formName"/>CustomData
                    Set updateAndReadData.Result = updateAndReadData.Data.updateData(key,formData)
            End Function
           
            Public Function deleteData(key as String) as ResultCode
                Dim curData as new <xsl:value-of select="$formName"/>CustomData
                Set deleteData = curData.deleteData(key)
            End Function
       
        End Class
       
        Public Class ResultCode
            Public Value as Integer 'We use HTTP values
            Public DBid as String 'Database and document we try our luck with
            Public DocID as String
        End Class
       
        Public Class ExtendedResult
            Public Result as ResultCode
            Public Data as <xsl:value-of select="$formName"/>CustomData
        End Class

Posted by Stephan H Wissel At 2:00 AM Location | Permalink | Comments (1) |      

Category Show-N-Tell Thursday
There is a current discussion going on if and how to use webQueryOpen Agents. Jake has some excellent examples what you can achieve, while Michel didn't find performance differences between an embedded view and a WQO agent. I was actually not surprised, since a single user would show any difference. My suspicion is that the results will change once you put a server under load. So I was looking around for a tool that anybody could use to run a little test (so Rational Performance Tester was out of the picture).
The search was short. Apache offers a tool called JMeter. I don't know how it stacks up against other frameworks and/or tools, but it does everything we need for the little test. What is also quite nice about JMeter: there is a Wiki and plenty of online information how to use it. Once you download the binaries and expand them into a directory, you are good to go.
JMeter is started as Java application using the jmeter.bat file. You also have the option to run a command-line or even start remote instances. To work with JMeter successfully you need to understand a few concepts. JMeter is hierarchically organized. JMeter executes TestPlans, TestPlans contain ThreadGroups and ThreadGroups contain Tests. A Test consists as a minimum of one or more sampler. You also want to add a listener to show your result graphically or as table. You have a lot of options to configure and script, so it will take quite a while until you can take advantage of all the functionality. However the good news: it takes only a few minutes to configure a simple load test like "Hammer this URL with 1000 users".

These are the steps:

1. Start JMeter
2. Right click on Test Plan: add Thread Group. Specify the number of users (a.k.a threads) and the ramp-up time. This is the time how long JMeter will take to open all the treads. Also you can specify the number of loops to run.
3. Right Click on the Tread Group and add a "HTTP Request HTTP Client" Sampler. Define the web server, port and relative URL (typically "/myNSF.nsf?OpenForm,  ...?OpenDocument or ...?EditDocument").
4. Right Click on Test Plan and add some listeners like the "Summary Result" or "Graph Result"
5. Save the file (Ctrl+S)
6. Run the plan (Ctrl+R)

That is all. You can look at the results in a tabular or graphic form, depending on the listeners you defined. Next step: run it against your favorite WQO.

Posted by Stephan H Wissel At 12:06 PM Location | Permalink | Comments (6) |      

Category Show-N-Tell Thursday
Continued from Part1.
Our application should start with a screen showing me only my documents and the option to switch to documents where I'm the approver. To do that I create a view based on the ($All) view and Just drag the Requestor column to the front.  I also sort the view descending by date, so at some time older entries will disappear

The first column is categorized. A simple click will do. The column shows the content of the field requestor:


We have the formula @Name([Abbreviate];Requestor) in the column. We could replace that with Requestor or leave it. It doesn't really matter. What matters is to use the same style later in the page when we embed the view. We will come back to the view in an instant to refine it. Now it is time to insert it into the page. I copy the content from the $$ViewTemplate for ($All) into a new page MyRequests. Doing that you will realize, that the field $$Viewbody doesn't get copied. A page can't hold fields.
To insert the view I select Create - Embedded Element - View:


The Properties need adjustment. So I select to use HTML and allow for 50 entries and page width (on the second tab).

Posted by Stephan H Wissel At 7:56 PM Location | Permalink | Comments (2) |      

Category Show-N-Tell Thursday
This is a follow-up post to an older thread on Notes.net. Configuring SSL and reading from remote locations can be a headache... unless you stand on shoulders of giants. Here are the steps that worked for me. While they are designed for R7 upwards with a JVM 1.4++ they also will work in R6 with the optional SUN SSL packages (just read the older post for configuration).

Update: The class didn't process HTTPPost correctly, so I updated the code, changes in bold.

What do you need:
1) Apache Commons HTTP Client
2) Apache Logging library (and codecs)
3) EasySSL Classes (EasySSL, EasyTrustManager)

Update (Thx John): Above links don't work anymore. EasyTrustManager can be found here and here. EasySLL here and here.
Traversing the broken URL above leads to the "readme pointing" to the new home called earth. A wildcard redirect would have been nice.

Once you have that a few simple lines of code will do. Note: you don't even need to configure SSL (but you SHOULD understand the security implications of NOT configuring it).
This post is also available on Notes.net. (or whatever it is called now).

Here is the class ...

Posted by Stephan H Wissel At 2:00 AM Location | Permalink | Comments (1) |      

Category Show-N-Tell Thursday
Great work is under way to make Domino work as a first class citizen in the shiny new Ajax world. With this field covered I decided to look the other way: how minimalistic can a Domino application get and still work properly. I want to build a small approval application, just one approver level. It is mainly for documentation, not to execute a sophisticated mapped out workflow. Something that is a little bit more structured that sending eMail requests.
So let us assume I know HTTP/HTML very well. I don't know JavaScript (and I might want to use the app with devices that don't support JavaScript). I don't know Java or LotusScript. I have a limited understanding of @Formula (some basic stuff like @Command([FileSave]) or @Trim(@ThisValue) or @if(...)). I also have a friend who does the CSS for me. I have an excellent understanding however how Domino does forms, how Author and Reader fields work and how Domino URLs are constructed (which resemble some kind of REST API, so learning them with my background in HTTP was quite easy). How far can I go?


There are quite some steps involved. But they are less than it looks like.

Step 1: Create the database


Step 2: Set the ACL

Since I want anybody to be able to use this little gem, I set default access to Author. Please --> In your environment follow best practices and set it to -No Access- and have an appropriate access group... but that is what the Administrator will do for you anyway. Just keep in mind to create the [Debug] role (You might not understand it but you liked the idea in that SnTT post).

Posted by Stephan H Wissel At 1:37 AM Location | Permalink | Comments (3) |      

Category Show-N-Tell Thursday
Slavek suggested in his comment to "The Eventful Notes Client" to use the Statusbar instead of popup messages. I replaced "@Prompt([OK];"Eventful Notes";" with "@Statusbar(", "MsgBox" with "Print" and "alert(" with "@window.status = (". Being naturally lazy I used Teamstudio Configurator for this. Here we go:

-- Open database into frameset --

07/09/2007 09:42:47 PM  Status Msg: Menu Page Title
07/09/2007 09:42:47 PM  Status Msg: Name of embedded Outline
07/09/2007 09:42:47 PM  Status Msg: $All Global Initialize
07/09/2007 09:42:47 PM  Status Msg: $All Initialize
07/09/2007 09:42:47 PM  Status Msg: $All QueryOpen
07/09/2007 09:42:48 PM  Status Msg: Initialize for database NotesEventExample
07/09/2007 09:42:48 PM  Status Msg: PostOpen for database NotesEventExample
07/09/2007 09:42:48 PM  Status Msg: $All PostOpen
07/09/2007 09:42:48 PM  Status Msg: $All OnSelect

-- Change to a different view --
07/09/2007 09:43:21 PM  Status Msg: $All QueryClose
07/09/2007 09:43:21 PM  Status Msg: $All Terminate
07/09/2007 09:43:21 PM  Status Msg: $All Global Terminate
07/09/2007 09:43:21 PM  Status Msg: SecondView Global Initialize
07/09/2007 09:43:21 PM  Status Msg: SecondView View Initialize
07/09/2007 09:43:21 PM  Status Msg: SecondView QueryOpen
07/09/2007 09:43:21 PM  Status Msg: SecondView PostOpen
07/09/2007 09:43:21 PM  Status Msg: SecondView OnSelect

Posted by Stephan H Wissel At 11:32 PM Location | Permalink | Comments (2) |      

Category Show-N-Tell Thursday
There have been many attempts to explain what Lotus Notes is. I'd like to add my own - from the perspective of software development:

Lotus Notes is an event engine that provides the developer with the capability to assemble an application from predefined events and react to them.

The languages available to react to events are: @Formula, LotusScript, JavaScript, Java, C/C++. The selection of available languages depends on the event you want to react on.
. Some examples of events are: Database opened, Default value of a field, Saving a document etc. There are MANY events and the knowledge how to use what event and when makes the Notes developer <g>
I assembled a small database with one outline, one page, one frameset, two views, one form, one subform and a few fields. I "mined" almost every event I could find with prompts to determine the sequence of the events firing (I didn't mine Hide-when formulas, the onKey..., onMouse... and onChange events).

After putting all the events into code I performed the following actions, starting with the empty database:

1. Open the database (into the frameset)
2. Switch from the View "$All" to the View "SecondView"
3. Create one document "EventSample"
4. Fill all fields with values
5. Press F9 to recompute
6. Change one more value
7. Press Ctrl+S to save
8. Close the document
9. Repeat Step 3-8
10. Select the View "$All"
11. Delete the current document (Del key)
12. Select the next document
13. Select the deleted document
14. Undelete the document
15. Close the database

I didn't play with agents, drag & drop or web documents (that would be more to add). There are a lot of events firing, see for yourself:

Update: Also checkout Part 2 of the story.

Posted by Stephan H Wissel At 2:00 AM Location | Permalink | Comments (2) |      

Category Show-N-Tell Thursday
I've been charged with delivering a workshop "Advanced Domino Web Development". Musing over the content of this workshop I realized, that you need just five things:

1. Understanding
2. Tools
3. Libraries
4. Reference
5. Code

In detail, there is a little more stuff to consider. Since a picture is worth a thousand words, here the details as MindMap:

To create this map I used a trial version of Tony Buzan's very own software.

Posted by Stephan H Wissel At 2:00 AM Location | Permalink | Comments (2) |      

Category Show-N-Tell Thursday
Theo Heselmans has a nice post about how to figure out if a user has a specific role without using @IsMember or @IsNotMember. He doesn't like @IsMember and @IsNotMember. While that is a question of taste (or speed?) it is a good idea to fully understand how the various comparisions work. = or *= work differnt with lists than @is(Not)Member. One of the hallmarks of @Formula is the ability to deal with lists (multiple elements, similar to arrays) in the various formulas. In a lot of the @Functions you can use lists ("red":"blue":"green") to compute your results. If a Notes item (a field) contains multiple values they are treated as lists in formulas. The following table shows the effect of the various comparison results. I limit myself to matches, you can play with "greater" and "less" comparisons.
Let us asume we have these lists:

list1 := "red":"yellow":"green"
list2 := "yellow":"red"
list3 := "black":"blue"
list4 := "red":"white"
list5 := "yellow"

Comparison	Effect	Matches for @true	Examples	Result
=	Compares every single value with the same position in the other list. If one list has less elements than the other list, then the shorter list is padded with the last value. In a list of one, that one element is compared with all others. If there is ONE match true is returned.	one match, same position	list1 = list2 list2 = list5 list5 = list1	@false @true @true
*=	Compares every member of the first with every member of the second list. If there is a single match true is returned. In difference to = the position doesn't play a role and there is no padding (needed)	one match, any position	list1 *= list2 list2 *= list5	@true @true
! [Expression]	Turns the expression into the opposite. Very clean. Should be always in front. For readability you might want to use brackets	depends on expression	!list1 = list2 !(list1 = list2) !(list2 *= list5)	@true @true @false
!=	not equal operator. Compares element by element, padds a shorter list with the last element (same as =). Returns true if one match pair is different. In other words: only returns false if both list have the same members in the same sequence.	one mismatch, any position	list1 != list2	@true
*!=	Compared every element from both lists and returns true if it finds one difference. It only will return false if both lists only contain one value (which can be there multiple times, that wouldn't matter)	if one list has 2 different values	"red":"red" *!= "red":"red":"red" list1 *!= list2	@false @true
@IsMember	Checks that all members of the first list are available in the second list. The sequence doesn't matter. If there is one element in the first list, that is not in the second list it returns false.	all, any position	@isMember(list2;list1) @isMember(list4;list1) @isMember(list5;list1)	@true @false @true
!@IsMember	reversal of @isMember. If there is just one element in the first list that is not in the second list it returns true.	one mismatch, any position	!@isMember(list2;list1) !@isMember(list4;list1)	@false @true
@isNotMember	Checks that none of the elements in the first list is in the second list. If just one element is there, but not others it already returns false. So you can't replace !@IsMember with @IsNotMember for lists arguments.	all mismatches, any position	@isNotMember(list2;list1) @isNotMember(list4;list1)	@false @false
!@isNotMember	Double negation to make your head spin <g>. Needs only a single match to return true. Equivalent to *=	one match, any position	@isNotMember(list2;list1) @isNotMember(list4;list1)	@true @true

Does your head spin now?

Posted by Stephan H. Wissel At 2:00 AM Location | Permalink | Comments (3) |      

Category Show-N-Tell Thursday
One of the new exiting features of the Eclipse RCP based Notes 8 client are themes. Themes allow you to change many visual aspects of the Notes client UI. While serious developers will take the time and effort to closely work with the corporate design department to define a unified user experience in all corporate colors and fonts, the hackers among us will be in for a quickfix. Here you go:

1. Locate the directory com.ibm.notes.branding ... inside <Notes Program Dir>\framework\shared\eclipse\plugins\
2. Create a backup of it, in case you screw it
3. Edit splash.bmp, add a lotus flower
4. edit themes/notes.css, look for mailtable>row>unread and change the color from Black to Red (yes it is back)
5. Do other wierd things to your hearts desire

Your mileage might vary!

Posted by Stephan H Wissel At 2:00 AM Location | Permalink | Comments (2) |      

Category Show-N-Tell Thursday
A short one today. A typical pattern for Notes and Domino forms is to have hidden fields at the beginning and end of a form. These hidden fields store lookups, IDs, relations and all sorts of things. As a unwritten convention these fields are marked red and hidden from reading and editing (basically all hidewhen options are checked). While developing or troubleshooting applications it is important that you can see the values of these fields. So instead of checking all hide options I use the following approach:
I create a hidden Computed For Display field of type number named "IsNotDebug". I put this formula in:
@IsNotMember("[Debug]";@UserRoles)
Then I use a formula in the hide when section: IsNotDebug. Is is kind of plain English: Hide when is not Debug.
By adding or removing the role [Debug] from your user-id you can switch on/off the display of the hidden fields. If you do a lot of local development, you might opt for a Environment variable instead of a role.
A final tip: To make things look better I often find hidden fields neatly put into tables with some explanations in the second column. This is not a good idea. Hide when formulas are executed separately for every paragraph in every cell. So you create a lot of extra computation. Best is to have the fields in a single paragraph (newlines with Shift-Control are ok). I use the pattern
Field : [the field] (Comment) | <-this could be just a bar or a new line.

Posted by Stephan H Wissel At 1:28 AM Location | Permalink | Comments (5) |      

Category Show-N-Tell Thursday
The title I wanted to give this post was "Spring clean your notes forms", but then it did get delayed....
One hallmark of Notes applications is their durability. It is quite common to find applications being in use, that root back into R3 or R4. Quite a number of these applications still look like that too. That's the flip-site of not needing to rip-and-replace. Sometimes you also will find, that older forms load slower and behave odd at times. Before you start beautifying your applications, it is a good idea to do some spring cleaning. To do so I use Domino's DXL to extract a form in XML, filter it through a XSLT transformation and reimport that into the database.
The LotusScript code is very straight forward, has been well covered on developer works and the help file and looks like this:

  Set importer = session.CreateDXLImporter(stream, dbCopy)
 importer.ReplicaRequiredForReplaceOrUpdate = False
 importer.DesignImportOption = DXLIMPORTOPTION_CREATE
 Call importer.Process

The "meat" is in the transformation. The XSLT file consist of 4 principal sections that contain templates. In section 1 we find the start of the output including the wrapper and calls to various templates. In section 2 design elements are filtered out, in section 3 design elements are tweaked, finally in section 4 the remaining DXL is copied 1:1 to the resulting document; . The sequence of the sections is not relevant, since XSLT uses priorities not sequence to determine what element to apply. What are the elements you can or should spring clean:

• In DXL I found NotesItems between the </body> and the </form> tag. Removing this from the form makes the form smaller, load faster without any change in behavior.
• Font information is encoded inside the <run> tag. The <run> tag is similar to HTML's <span>. When you designed and re-designed forms over and over, there will be <run> tags that only contain a font change, but not actual any characters. Filtering that out lightens the form
• Filter out all paragraph formats and/or fonts to be able to apply a new look & feel more easily
• Convert access controlled sections into subforms (to work in the web)
• Remove (all) LotusScript code and move it to libraries (see an upcoming SnTT post on "Classical Forms" about that).

Let us look at some code....

Posted by Stephan H. Wissel At 2:00 AM Location | Permalink | Comments (2) |      

Category Show-N-Tell Thursday
Notes 8 is build using Lotus Expeditor which is an Eclipse RCP application. So theoretically you should be able to load your own plug-ins (which might not even be Notes related). Of course this is utterly not supported <g>. The functionality to point to an Eclipse Updates site is by default switched off. To switch it on locate this file:
<notes install dir>\framework\rcp\plugin_customization.ini

Add one line:
com.ibm.notes.branding/enable.update.ui=true

and restart the Notes client. You will get access to the standard Eclipse Update site dialog here:



It would be interesting to hear what plug-ins work (Azureus anyone)

Posted by Stephan H Wissel At 12:00 PM Location | Permalink | Comments (4) |      

Category Show-N-Tell Thursday
Domino Designer allows you to pick whatever form name you deem fit. It can contain letters, numbers, spaces, special characters. However when you develop for the web you might want to be a little more picky with what you use. Domino renders the form name into the HTML form name when showing the form in the browser. It also generates some JavaScript referencing this form. The form "Memo" is translated into "_Memo", "Response" is translated into "_Response". However "9. Market Survey" or "action.7" or "Bla Blub" all get translated into "_DominoForm". So if you have any logic that relies on the form name, you want to make sure the form name you pick can serve as a JavaScript variable name too.
If you retrofit an existing application, you might want to use form alias to achieve that. And for replacing that form name everywhere else there is Teamstudio Configurator.

Posted by Stephan H Wissel At 1:00 AM Location | Permalink | Comments (1) |      

Category Show-N-Tell Thursday
Domino 7 allows you to create web services easily. You can start with a WSDL file and have Domino generate the Stub Classes for you or you start with the Class File and get a ready made WSDL file. Unfortunately both approaches require that you code quite a bit. Domino doesn't provide a ready baked set of SOAP web services out of the box. You do have a kind of REST web service using ?ReadViewEntries, but that is read ony.
A very likely candidate for a web service is the creation, update and deletion of documents with a given form (commonly refered as CRUD). Domino allows you to store any field in a document, even if it hasn't been defined anywhere making it very flexible and so difficult to understand if you are used to an RDBMS.
It would be fairly easy to design a web service (after Julian has enlightened you here, here and here), that allows an arbitrary number of field value pairs to be stored in a document. However this would defeat the principle of clear cut interfaces. More appropriate is to limit the possible input to the input fields defined in a form and giving back all fields including the computed ones. How to get there?

1. Export a domino form as DXL
2. Transform the DXL into WSDL using XSLT
3. Import the WSDL into a Domino Web Service

Creating the WSDL file first instead of Webservice code has a number of reasons. First it follows the "contract first" programming patterns, where the WSDL file is the "contract" and code has to stick to that interface, secondly it allows you to decide whether to use LotusScript or Java to implement the service. Last not least: Currently you can't easily export/import a web service design element in DXL.

Posted by Stephan H Wissel At 1:00 AM Location | Permalink | Comments (1) |      

Category Show-N-Tell Thursday

doc.ComputeWithForm seems to be confusing for a lot of developers. The Designer Help states: Validates a document by executing the default value, translation, and validation formulas, if any are defined in the document form. No word about computed Fields. In Notes 4.6 they didn't work, which still can be a problem for some. Starting with R5 they compute even if sometimes it doesn't look like. Also some challenge and an odd behavior remains.

In a nutshell these are the facts you need to keep in mind when using doc.ComputeWithForm:

• For editable fields the Default, Input Translation and Input validation fields are executed. The Default formula is executed when the field does not exist in the document. Typically that is only the case for a new document, but can be true also when the item was removed through code or the field was added to a form later
• For computed fields the formula is executed
• For computed when composed fields the formula is executed if the item does not exist in the document (same rules apply as for the default formula)
• Computed for display formulas are NOT executed. Therefore any formula depending on a field value of a computed for display field will take the value as empty thus producing unexpected results
• Side effects in Formulas like FIELD xxx := or @SetProfileField are executed. @Commands are not. Also @DeleteDocument has no effect.
• Self referencing fields are problematic. A Formula like @if(@thisValue="";1;@ThisValue+1) results in values 2-4-6-8 etc. You might change your formula to this: @If(@ThisValue="";1;@IsDocBeingEdited;@ThisValue+1;@ThisValue) and use LotusScript to increment your counter manually
• @IsDocBeingEdited returns @False, so you can influence if a formula is executed in the UI or with ComputeWithForm. A standard blocker for fields would be: @if(@IsDocBeingEdited;"";@Return(@ThisValue)).
  Memento: You can't use that for display only fields since they MUST compute when you open a form, while all other formulas don't compute in read mode (OK: Defaults for new fields compute [but are not saved] as does the Window title)
• You might want to use a special form to use with ComputeWithForm. There you can do all sorts of data massage without writing lengthy LotusScript code. The code snippet would look like this:
  oldForm = doc.Form(0) doc.Form = "SpecialComputeForm"
  Call doc.ComputeWithForm(false,false)
  doc.Form = oldForm
  This also allows you to ban evaluate() from your script code thus keeping LotusScript and @Formula strictly separate
• None of the Form events are fired. So no LotusScript, JavaScript or @Fomula in Form events (QueryOpen, QuerySave etc.) execute. Is is @Formulas in fields only
• When your field returns more than 15k of data the summary flag is not set, so you can't use it in a view column. This is a bug that has been reported as SPR# TNIT5EYJ9N. Didn't find it in any release notes yet.

That pretty much sums it up.

Posted by Stephan H Wissel At 12:04 PM Location | Permalink | Comments (3) |      

Category Show-N-Tell Thursday
When cleaning up a domain and reconfiguring servers, domains and routes one critical success factor is the distribution of changes to all servers. In a well maintained domain the replicator task will take care of that. However that might take hours and if the domain you are working on is well maintained, there is little need for a clean-up in the first place .
To be independent from replication schedules and other quirks I got an agent at hand that helps me there. You can copy that agent to your Domino Directory and take a few easy steps:

1. Create a local replica of the Domino directory
2. Copy the agent into it
3. Make sure you have a Notes client that can reach ALL servers
4. Make sure you have enough otherwise to do (or run this agent from a VMWare)

The code is pretty simple.

Posted by Stephan H. Wissel At 2:00 AM Location | Permalink | Comments (5) |      

Category Show-N-Tell Thursday

By now you should be able to extract your database design as DXL. This opens a series of possibilities to run reports against your database to gain more insights into its design. Since Notes is around for so many years you will face the situation to inherit a database you haven't written. To understand its logic you need so see how Hide-when formulas have been used in the