I have spent many years working deep within the Notes Indexing Facility (NIF). I’m not done, because there is more to be done and harvested from the rich functionality well-bundled from day one by the earliest Domino developers. It can be a challenging area because so much variety of data flows through the code and onto the glass. If one is not careful, you will make one user story a delight while another becomes quite less so. 

One of the most problematic operations in NIF is its handling of updates. Someone could say “Well, what is the problem? Other database products have no issue updating their indexes, why is Domino any different?” The answer lies in the ease with which views and folders are created in Designer. It is a very common story that views constantly grow in number and complexity. 

So, I often reply to the question of the difference this way: “If you went to your MySQL or Oracle or DB2 or MongoDB administrator and asked for 200-1000 indexes on your database, what would they say?” Of course, they would laugh you out of the room, as they should. Yet, that number of indexes is quite normal on a Domino database and it is why they are updated lazily, on a scheduled or as triggered or forced. There are a variety of update/refresh rules and I will not go into them here. 

Some of the costliest issues with NIF are due to excessive queueing for updates. Most of the APIs and documentation call it “refreshing” when a user opens a view and wants to see the latest updates, which is completely reasonable. The trouble what when a large number of users (well, threads) do that at once, they each compete for the right to update the view in a “fair read/write” queue – that is, those who only want to read get no preference over writers (we don’t starve updates). The result can be a breach in Service Level Agreement (SLA) timings and opening a view simply takes too long. 

Following is a diagram I put together for SPR JCUS8MXLA2. Included in the mix is the “Update Task” representing scheduled, dedicated view updating: 

You may or may not know that in the v9.01 feature packs, Domino development rather quietly produced a feature to attack this problem. It is called “inline view updating” or just “inline.” You can toggle it on via

load updall <database name> [-T <viewname>] -inline on/off

What this does is to apply view updates at time of document updating. All refresh requests at view open time can therefore be treated as nops, as they are. Document updating becomes a heavier unit of work, but the view open and read performance trade-off is worth it.

A second effort was launched in Domino V10, which detects and then automatically creates a dedicated update thread to high usage views. It is enabled via NIF_VIEW_USAGE_ENABLED=1 in notes.ini.

I wanted to measure the effect of the first effort, -inline, so this summer at HCL Chelmsford (well, at home due to COVID-19) I’ve had the privilege of working with an intern in core Domino, Joseph Calles. I set Joseph on a quest to provide real timings with -inline on and off. He wrote some scripts that allowed him to vary the read vs update load against a 200K document database with three views and collected stats using “show trans” at the Domino console. Joseph capped the document update and view operations at 10/second, but of course your numbers will scale according to volume, bulk of data and hardware utilized.

No one is claiming these numbers are real-world, the data and processing is automated. But they are consistent relative to the gathered set. And they show some trends people should know about as they consider using -inline. The variance in test load was simple: for each run, the number of all operations was incremented by one per second.

The OPEN_COLLECTION transaction is what bogs down trying to update a view when inline is off. The benefit is very clear to see in the first graph, and it should be noted that without inline, time to open a view increases much faster than linearly. That .. is the log jam I mentioned.

In the second graph, open view and read view entries are combined to provide a single duration. This is close to what a user would experience opening a view and getting his/her first window of entries rendered. Again, the spike with larger numbers of transactions without -inline is clear.

Now, very few SLAs will be violated with 800 millisecond response. Again, the point is the curve. Showing maximum time to open a view shows the erratic, worst-case user experience, which would indeed violate most SLAs (note, times are in seconds in this graph):

Truth in advertising — document update times do increase with -inline on, but you would expect them to. And they don’t increase exponentially but linearly. But don’t confuse this with a decrease in throughput — it is not affected, only time to complete:

This final graph treats all operations — update, finds, view opens and view entry reads equally and averages their combined duration. So, overall duration impact of -inline processing if you will:

But it’s important to remember the original use case — multiple users trying to open and read a view but first “refreshing” (that is, updating). With -inline on, the refresh operation is unnecessary since any view being read is updated during a NSFNoteUpdate operation. So those view open operations are much lighter weight.

To say it again, this behavior is “normal” for transaction-centric database systems, especially the relational variety where data lives in neat rectangles (ok, they have progressed a bit). But in Domino, a document-storing and -processing NoSQL engine that supports so many different indexes constructing index data so many different ways, a lazy update model is appropriate lots of the time. And the -inline feature addresses those cases where it is not appropriate.

So, if you have contentious views — and please, not all views are hotly contended for — I recommend you strongly consider turning on inline view updating, using

load updall <database name> -inline on -T <name of contentious view>

And reap good benefit.

I would beware of turning it on for all views in a database as the document update cost could then surge. You know your most contentious views; tackle those first.

Comment wrap
Further Reading
Digital Solutions | September 15, 2020
Project Eleven: Get the Most Out of Domino with Easy Direct Upgrades
HCL is fully committed to enabling you and your organization to maximize the value you get from the Domino and Sametime platforms today and in the future.   In the latest release of Domino v11.0.1, we’ve delivered innovative new capabilities such as low-code app dev and integration with Microsoft Active Directory. We’ve also launched a brand new Sametime meetings solution over the last two years. The response from customers and business partners has been unanimously positive!    Today, we’re excited to announce, “Project Eleven,” a new program designed to help every Domino and Sametime customer running on releases prior to v11 have a smooth upgrade experience. As part of the HCL Digital Solutions Academy, Project Eleven’s goal is to enable you to:   Go straight to Domino v11.0.1: HCL now supports direct upgrades from any of Domino version prior to v11. Whether you’re on v8, v9, or older, you can go straight to v11.0.1 with no additional steps necessary. Experience the latest version of Sametime risk-free: Secure meetings and data privacy are the most important features in today’s environment. Sametime meetings runs on Cloud Native technologies allowing you to auto scale as needed. Experience our brand new, highly secure, scalable video meetings and enterprise chat risk-free. Augment your skills today: Attend our online training sessions and get best-in-class support and training from a technical advocate in your local language. We’re rolling out more than 40 webinars and workshops and new articles, cookbooks, and help guides. Speak to a technical expert: Schedule an appointment with one of our highly skilled technical advisors to understand your options and create an upgrade plan that’s right for you.    Check out the new Project Eleven webpage to find all the resources you need to make your upgrade worry-free.                                             
Digital Solutions | September 8, 2020
Introducing HCL Domino Early Access Program
We’re proud to announce the HCL Domino Early Access Program, a new program that gives our customers and business partners a chance to test new product features early in the development cycle. This program provides a preview of features and functionality that may be shipped with a future version of HCL Domino and gives us the ability to incorporate your valuable feedback. At this stage, we are providing early access to features that contribute to HCL Domino v12.  The Early Access Program aims to get feedback on specific features, with a more formal traditional beta program planned at a later stage to receive feedback on the overall release quality and language packaging.  The Early Access Program is feature-driven and iterative, meaning new iterations and code drops will be made available as soon as it is declared ready for testing by our development team. New code drops can be provided in just a few weeks after the previous one, on short notice, without sticking to a fixed release cycle.   Based on your feedback we may proceed, improve or adjust the feature, or in some cases withdraw the feature from the code stream. The following graphic illustrates this concept: Each code drop is provided in the form of a Docker image which enables us to ship new iterations faster and to minimize the time to upgrade from one code drop to the next. You can expect us to deliver code drops at a fast pace as we want participants to have exclusive access to our latest product feature set. Please note that only the newest code drop will be available for you to download. How to Participate HCL Ambassadors who have registered Domino v11.0.1 Beta Preview will automatically be added to Domino Early Access Program. Customers and partners with a current Notes/Domino entitlement will also have access to the Early Access Program as of today. There is no need to register separately to participate. For overall...
Digital Solutions | September 3, 2020
New Release: HCL SafeLinx 1.1, – Secure VPN Services for Cloud or On-Premise
At HCL, we understand that securing your network is a topmost priority. HCL SafeLinx provides remote users with secure web-based and client-based access to enterprise applications and today, we’re excited to announce that the latest release of HCL Safelinx 1.1. For Domino customers using HCL Nomad, SafeLinx v1.1 enables you to securely access your Domino apps on a phone, tablet, or desktop browser without the need of an additional VPN. Introduction of HCL SafeLinx SafeLinx uses standards-based protocols to enable secure access from mobile computing devices outside the firewall to business applications and data on your organization's internal network. SafeLinx supports an extensive list of standard Internet Protocol (IP) and non-IP wireless bearer networks, server hardware and operating systems, and mobile security protocols. Remote users can connect to your internal network from a range of desktop, notebook, and handheld mobile devices and operating systems. Top Product Features Client and clientless access Network choices: Supports a worldwide selection of wireless and wireline network technologies, enabling a worldwide scope of connectivity. Cellular, LAN, satellite, dial-up, radio (IP, non-IP) Authentication: Multiple strong authentication options (Radius, Secure ID, LDAP, Active Directory, Certificate) Encryption DES, Triple DES, RC5, AES and FIPS 140-2-certified Network Optimization and Scalable and Reliable Customer Benefits Supports all major platforms: Windows, Linux, MacOS, iOS, and Android Best enterprise VPN with reverse proxy capabilities Mobility clients and Distributed Administration - Gatekeeper Dedicated mobile app for Android and iOS Two-way secure authentication Supports seamless, non-disruptive cross-network roaming Quick and easy deployment of servers and agents What's New in HCL SafeLinx 1.1? This new release includes improved functionalities of VPN and reverse proxy and support for TLS 1.3. View documentation on SafeLinx 1.1 here. For all Notes Domino Complete Collaboration cCustomers, SafeLinx v1.1 is now available as a free entitlement and will be listed...
a/icon/common/search Created with Sketch.