I have spent many years working deep within the Notes Indexing Facility (NIF). I’m not done, because there is more to be done and harvested from the rich functionality well-bundled from day one by the earliest Domino developers. It can be a challenging area because so much variety of data flows through the code and onto the glass. If one is not careful, you will make one user story a delight while another becomes quite less so. 

One of the most problematic operations in NIF is its handling of updates. Someone could say “Well, what is the problem? Other database products have no issue updating their indexes, why is Domino any different?” The answer lies in the ease with which views and folders are created in Designer. It is a very common story that views constantly grow in number and complexity. 

So, I often reply to the question of the difference this way: “If you went to your MySQL or Oracle or DB2 or MongoDB administrator and asked for 200-1000 indexes on your database, what would they say?” Of course, they would laugh you out of the room, as they should. Yet, that number of indexes is quite normal on a Domino database and it is why they are updated lazily, on a scheduled or as triggered or forced. There are a variety of update/refresh rules and I will not go into them here. 

Some of the costliest issues with NIF are due to excessive queueing for updates. Most of the APIs and documentation call it “refreshing” when a user opens a view and wants to see the latest updates, which is completely reasonable. The trouble what when a large number of users (well, threads) do that at once, they each compete for the right to update the view in a “fair read/write” queue – that is, those who only want to read get no preference over writers (we don’t starve updates). The result can be a breach in Service Level Agreement (SLA) timings and opening a view simply takes too long. 

Following is a diagram I put together for SPR JCUS8MXLA2. Included in the mix is the “Update Task” representing scheduled, dedicated view updating: 

You may or may not know that in the v9.01 feature packs, Domino development rather quietly produced a feature to attack this problem. It is called “inline view updating” or just “inline.” You can toggle it on via

load updall <database name> [-T <viewname>] -inline on/off

What this does is to apply view updates at time of document updating. All refresh requests at view open time can therefore be treated as nops, as they are. Document updating becomes a heavier unit of work, but the view open and read performance trade-off is worth it.

A second effort was launched in Domino V10, which detects and then automatically creates a dedicated update thread to high usage views. It is enabled via NIF_VIEW_USAGE_ENABLED=1 in notes.ini.

I wanted to measure the effect of the first effort, -inline, so this summer at HCL Chelmsford (well, at home due to COVID-19) I’ve had the privilege of working with an intern in core Domino, Joseph Calles. I set Joseph on a quest to provide real timings with -inline on and off. He wrote some scripts that allowed him to vary the read vs update load against a 200K document database with three views and collected stats using “show trans” at the Domino console. Joseph capped the document update and view operations at 10/second, but of course your numbers will scale according to volume, bulk of data and hardware utilized.

No one is claiming these numbers are real-world, the data and processing is automated. But they are consistent relative to the gathered set. And they show some trends people should know about as they consider using -inline. The variance in test load was simple: for each run, the number of all operations was incremented by one per second.

The OPEN_COLLECTION transaction is what bogs down trying to update a view when inline is off. The benefit is very clear to see in the first graph, and it should be noted that without inline, time to open a view increases much faster than linearly. That .. is the log jam I mentioned.

In the second graph, open view and read view entries are combined to provide a single duration. This is close to what a user would experience opening a view and getting his/her first window of entries rendered. Again, the spike with larger numbers of transactions without -inline is clear.

Now, very few SLAs will be violated with 800 millisecond response. Again, the point is the curve. Showing maximum time to open a view shows the erratic, worst-case user experience, which would indeed violate most SLAs (note, times are in seconds in this graph):

Truth in advertising — document update times do increase with -inline on, but you would expect them to. And they don’t increase exponentially but linearly. But don’t confuse this with a decrease in throughput — it is not affected, only time to complete:

This final graph treats all operations — update, finds, view opens and view entry reads equally and averages their combined duration. So, overall duration impact of -inline processing if you will:

But it’s important to remember the original use case — multiple users trying to open and read a view but first “refreshing” (that is, updating). With -inline on, the refresh operation is unnecessary since any view being read is updated during a NSFNoteUpdate operation. So those view open operations are much lighter weight.

To say it again, this behavior is “normal” for transaction-centric database systems, especially the relational variety where data lives in neat rectangles (ok, they have progressed a bit). But in Domino, a document-storing and -processing NoSQL engine that supports so many different indexes constructing index data so many different ways, a lazy update model is appropriate lots of the time. And the -inline feature addresses those cases where it is not appropriate.

So, if you have contentious views — and please, not all views are hotly contended for — I recommend you strongly consider turning on inline view updating, using

load updall <database name> -inline on -T <name of contentious view>

And reap good benefit.

I would beware of turning it on for all views in a database as the document update cost could then surge. You know your most contentious views; tackle those first.

Comment wrap
Further Reading
Digital Solutions | November 16, 2020
Updates Regarding HCL Notes v11 on Apple Big Sur
Apple officially released macOS Big Sur, its latest operating system, on Nov 12. We are aware that customers were unable to run the HCL Notes v11/11.0.1 client on a Mac after upgrading. Various fixes have been applied to allow HCL Notes to run on Big Sur. You can read our tech note here. This has all been resolved. The fix pack is available on Flexnet.
Digital Solutions | November 5, 2020
Remotely Debugging Java Applications with Domino 11
For Domino customers on v9 or 10, you might be familiar with debugging a Java agent remotely on IBM JVM using the following server ini’s and set Port 8000 in Debug Configuration from the Designer client: JavaEnableDebug=1 JavaDebugOptions=transport=dt_socket,server=y,suspend=n,address=8000  This will work up to Domino v10 but not on Domino v11. In fact, doing so will cause it to crash! This article is going to help developers looking for a way to debug Java agents remotely on Domino v11. The pre-requisites are:  Domino v11 or above on Windows Server 2016 / 2019. HCL Domino Console. Notes, Designer and Administrator Client v11 or above. Java agent to be debugged. Any Web Browser to run Java agent.  Domino v11 uses OpenJDK. With the above ini's enabled, both the JVMs would attempt to bind to same port (8000) and the second process that attempts to bind to this port would fail and crash. To overcome this problem, the idea is to remove the Port 8000 from the Debug and attach it to some other random Port.  First, you will need to add the below ini’s to the server’s notes.ini.   JavaEnableDebug=1  JavaDebugOptions=transport=dt_socket,server=y,suspend=n  Note that we are not binding the Debug to Port 8000 by omitting address=8000, as compared to ini which was set on earlier version of Domino.  The JVM will then pick a random port. To know the random Port number, simply quit and load http or run your Java agent from web browser and check the console from HCL Domino Console Program. You will need to restart http task after adding those ini’s to make it work.  Note down this Port number somewhere as you would need it in next step. (The server console in the Administrator Client doesn’t show the Port number information.) With your Port number, you are going to bind to the Debug from the Designer client. Go to Tools menu > Debug Server-side Javascript > Manage Debug Configuration You should land on the Debug Configuration window. Here you need to mention the Port number which you noted down from the HCL Domino Console...
Sending future mails from your HCL Notes Client
Have you ever wanted to schedule future emails from your Notes Client? Well, now HCL Domino supports scheduling email messages, which allows you to do just that. So, draft your email, assign it a send date, and sit back and relax. The email will be delivered by the router at the time you specify. This feature is very useful for scheduling announcements, newsletters, or any mail which you want to be delivered at a future specified time. You need to be on Domino v10 or v11 (using the Mail10.ntf or Mail11.ntf template) to be able to schedule messages to be sent in the future. Here’s how it works. The configuration document setting “Allow users to schedule a delivery time for messages” is enabled by default, so you don’t have to change anything in that document. You would make sure that the mail policy setting has the checkbox selected to enable the delivery option, as illustrated here. If you don’t have a mail policy, then you can create a new one. If you already have a mail policy, you merely update it. After creating or updating the mail setting, you need to give the following command to update the mail policy in the user’s mail file. Tell adminp process mail policy The next step is to restart the Notes Client, so that it will pull the new policy settings from the server. Advanced users can use this command from the command prompt of their client machine to update the policy on the client's side without restarting: C:\notes\ndyncfg -2 Once the settings are pulled by the Notes Client, you will see the following option in delivery options while sending mail: “Delivery Options at the user side.” Simply select the date and time for delivery and the router will deliver it at that...
a/icon/common/search Created with Sketch.