Processing large amounts of data in CRM, using Deluge scripts, isn’t easy. In a perfect world, a CRM function would run as long as it needs to and you would just page your requests, like this:

for each page in {1,2,3,4,5,6….}

{

For each rec in zoho.crm.getRecords(“Module X”,page)

{

// do stuff

}

}

The above works fine if your whatever your “do stuff” part of the function is really fast. Sadly, this is usually not the case, and more often than not Zoho craps out after too many minutes of processing. One solution to this is outlined below.

CRM Workflow Iterator

I call the technology described below a Workflow Iterator. In general, it consists of a custom module, a workflow function that is executed when a record of the custom module is deleted, and a standalone function that is called by the delete workflow.

The Workflow Iterator Module

The module is simple. It requires a URL field, an integer field called Page, an integer field called Count, and an integer field called Max Pages. Each of these fields should be required.

Page will be used by your final function as the page parameter in functions such as zoho.crm.getRecords, or zoho.crm.searchRecord. Count serves the same purpose.

URL will be the REST API function name of your standalone CRM function that processes your data.

Workflows

Once you have created the module you will want to add a button to it, visible on the View page.  You can call the button whatever you want, but it will be used to generate multiple copies of a single created module record. The function the button calls is as follows.

iterator = zoho.crm.getRecordById(“Workflow_Iterator”,id.toLong());

counter = 1;

page = iterator.get(“Page”);

for each  i in {1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30}

{

imap = Map();

imap.put(“URL”,iterator.get(“URL”));

imap.put(“Count”,iterator.get(“Count”));

imap.put(“Page”,page + 1);

page = page + 1;

if(counter <= iterator.get(“Max_Pages”))

{

zoho.crm.createRecord(“Workflow_Iterator”,imap);

}

counter = counter + 1;

}

Because Deluge doesn’t have a for-next loop, you will have to decide the maximum number of records you want to create by extending the array in the for each i in {…} line.

Once this function is in place you will need to create a workflow that is executed when any of the Workflow Iterator records are deleted. The workflow is simple.

You will create a custom function that is run on the delete event. As seen in the image below it should pass three parameters, page, count, and URL.

The actual function is simplicity itself:

res = invokeurl

[

url :url

type :GET

parameters:{“page”:page,”count”:count}

];

The Function

The guts of the CRM Workflow Iterator will be your function that processes your data. Here is a stub to give you an idea of how it works.

try 

{

for each  record in zoho.crm.getRecords(“Workflow_Iterator”,page,count)

{

info record.get(“id”);

}

iterator = zoho.crm.getRecords(“Workflow_Iterator”,1,1);

if(iterator.size() != 0)

{

deleteRecordMap = Map();

deleteRecordMap.put(“module”,”Workflow_Iterator”);

deleteRecordMap.put(“id”,iterator.get(0).get(“id”).toString());

deleteResp = zoho.crm.invokeConnector(“crm.delete”,deleteRecordMap);

}

}

 catch (Exception)

{ //send email to someone about the issue

}

return “”;

I highly recommend you embed your entire function in a try/catch block, just in case. Then after your loop comes the magic. Fetch the first Workflow Iterator record, and if there are any left, delete it. Viola, this kicks off the delete workflow, which calls this function, ad nauseum until all of the Workflow Iterator records are deleted.

It is critical that the processing part of this function is robust. Whatever you do in your processing you should be sure there are no dependencies on the order in which the records are processed, or how many times they are processed. This is because you are not guaranteed any order.

Once your function is ready to run, get the REST API value and put it aside.

Running the Iterator

Once you’ve created all of the pieces, it’s probably best to give it a quick test before actually processing any of your other data. Create 1 new Work Iterator record and copy the URL you saved in a previous step into the URL field. Fill in the other fields, giving Max Count a handful of records. Save the record.

From the record View click the button you created to duplicate it. Finally from the List view, choose a record and delete it. This should kick of the function/workflow cycle that will ultimately delete all the records.

If you have any ideas or suggestions to improve this, please feel free to reach out.