This article reports another example of interaction between SOLR delta index and data load process.

In particular, If we have to introduce the delta index- in place of the full- and the data load is already set up, or we have to plan from scratch the data load process, we should keep in mind that the data load operations, the order they are executed and the delta updates recorded in the temporary delta tables (TI_DELTA_CATENTRY and TI_DELTA_CATGROUP) are strictly connected; we should not consider the data load a black box respect to the SOLR index, when we need to update the system through the delta mode. The SOLR index and the Data Load are unfortunately tied in a way we cannot think to build a loading process without knowing in which way they interact.

I report two specific cases: 

1. delete bundles (reported in the previous article);
2. delete sales categories;

This topic's title could seems a bit alarming but it's just a pinch of irony I would give, most of all because I struggled for so many weeks on this feature that it was like fighting against the "dark". However, I just want to focus on few things we should think about before to plan the introduction of the delta index when the environment is updated through the data load.

In particular, if we have to introduce the delta index- in place of the full- and the data load is already set up, or we have to plan from scratch the data load process, we should keep in mind that the data load operations, the order they are executed and the delta updates recorded in the temporary delta tables (TI_DELTA_CATENTRY and TI_DELTA_CATGROUP) are strictly connected; we should not consider the data load a black box respect to the SOLR index, when we need to update the system through the delta mode. The SOLR index and the Data Load are unfortunately tied in a way we cannot think to build a loading process without knowing in which way they interact.

Ok, let's leave fancy words! I will report two specific cases: 

1. delete bundles;
2. delete sales categories (second part of this article);

One of the most annoying thing with the previous version of WCS (until de v7 FEP5) was the impossibility to perform a delta build of SOLR index after a data load process also for few line changes  ... basically a wast of time and resources.

However, FEP5 has finally introduced the possibility to make it works. In fact, new mediators have been implemented to load the data load changes in the temporary index tables and than execute successfully a Delta build reindex.

To test this interesting and very useful feature I realized a specific use case with Aurora on Windows platform, WCS Toolkit Enterprise v7 FEP6, DB2 and SOLR.

The test case:

1. Create a simple CSV file with the introduction of a new product: "Fancy Laptop"
2. Configure the data load (not using the new mediators) to load this product
3. Run the data load
• test: the product does not appear in the Front End
4. Run a DELTA rebuild of the SOLR index
• test: the product does not appear in the Front End
5. Run a FULL rebuild of the SOLR index
• test: the product appears in the Front End
6. Update data load configuration (adding the use of the new mediators and properties)
7. Change the product's name in the CSV ("New Fancy Laptop")
8. Run the data load
9. Run a DELTA rebuild of the SOLR index
• test: the product appears in the Front End