Multiple Request Merge Example

There is a limit on how much data can be sent to a Cloud Office Print server at once. Let's say you have a template for product specifications and you want to generate one merged PDF file for 50.000 different products. It is possible that you cannot send all the data for all the products at once to the Cloud Office Print server. In this example we will show you how you can split one big merge request into multiple smaller merge requests.

Template

A simple template will be used since the goal of this example is to show how you can split one big merge request into a few smaller ones. The template will contain one simple tag {test}. Tags are used in a template as placeholders to let the Cloud Office Print server know what needs to be replaced by data. In this case, the simple tag {test} will be replaced by whatever value is given to the Cloud Office Print server for the tag with key 'test'. In this example we use a template with filetype docx, but this can be any of the allowed template types (see here).

Code (SDK)

NOTE: For an overview of all the possibilities of this SDK, we refer to the documentation on our website. First we create a new file and import the Cloud Office Print library:

import cloudofficeprint as cop

const cop = require('cloudofficeprint');

import * as cop from 'cloudofficeprint';

import com.CloudOfficePrint.*;

Then we need to set up the Cloud Office Print server where we will send our template and data to:

SERVER_URL = "https://api.cloudofficeprint.com/"
API_KEY = "YOUR_API_KEY"  # Replace by your own API key

server = cop.config.Server(
    SERVER_URL,
    cop.config.ServerConfig(api_key=API_KEY),
)

const SERVER_URL = 'https://api.cloudofficeprint.com/';
const API_KEY = 'YOUR_API_KEY'; // Replace by your own API key

const server = new cop.config.Server(
    SERVER_URL,
    new cop.config.ServerConfig(API_KEY),
);

const SERVER_URL = 'https://api.cloudofficeprint.com/';
const API_KEY = 'YOUR_API_KEY'; // Replace by your own API key

const server = new cop.config.Server(
    SERVER_URL,
    new cop.config.ServerConfig(API_KEY),
);

Server copServer = new Server("https://api.cloudofficeprint.com/");
copServer.setVerbose(true);
copServer.setAPIKey(APIKey);

If you have a Cloud Office Print server running on localhost (e.g. on-premise version), replace the server url by the localhost url: http://localhost:8010

We also need to create the main element-collection object that contains all our data. Let's say we have 100 different customers for who we need to fill in the template and we want to merge the resulting files into a PDF. In this example, we are just going to repeat the property 'test' with value 'test' 100 times, but normally you would have different data for each customer.

data = {"file" + str(i): cop.elements.Property("test", "test") for i in range(100)}

const data = {};
for (let i = 0; i < 100; i += 1) {
    data[`file${i}`] = new cop.elements.Property('test', 'test');
}

const data: { [key: string]: cop.elements.Element } = {};
for (let i = 0; i < 100; i += 1) {
    data[`file${i}`] = new cop.elements.Property('test', 'test');
}

Hashtable<String, RenderElement> data = new Hashtable<String, RenderElement>();
for (int i = 0; i < 100; i += 1) {
    String key = "file" + i;
    data.put(key, new Property("test", "test"));
}

We want the output PDF files to be merged, so we create an output configuration:

config = cop.config.OutputConfig(
    filetype="pdf",
    pdf_options=cop.config.PDFOptions(
        merge=True,
    ),
)

const config = new cop.config.OutputConfig();
config.filetype = 'pdf';
config.pdfOptions = new cop.config.PDFOptions();
config.pdfOptions.merge = true;

const config = new cop.config.OutputConfig();
config.filetype = 'pdf';
config.pdfOptions = new cop.config.PDFOptions();
config.pdfOptions.merge = true;

PDFOptions pdfOpts = new PDFOptions();
pdfOpts.setMerge(true);
Output conf = new Output("pdf", "raw", "libreoffice", null, null, pdfOpts, null);

Let's assume that the Cloud Office Print server can't handle all the data at once, so we need to split our data into multiple requests. Let's use 10 requests with each 10 elements in the data (a total of 100 data elements).

output_files = []
for i in range(10):
    # Create print job with 10 data elements
    print_job = cop.PrintJob(
        data={
            # Select 10 data elements from the data
            k[0]: k[1]
            for k in list(data.items())[i * 10 : (i + 1) * 10]
        },
        server=server,
        template=cop.Resource.from_local_file("template.docx"),
        output_config=config,
    )

    # Execute the print job and save the response to a list
    output_files.append(print_job.execute())

(async () => {
    const outputFilesProm = [];

    for (let i = 0; i < 10; i += 1) {
        // Create print job with 10 data elements
        const dataSplit = {};
        Object.entries(data)
            .slice(i * 10, (i + 1) * 10)
            .forEach(([key, value]) => {
                dataSplit[key] = value;
            });
        const printJob = new cop.PrintJob(
            dataSplit,
            server,
            cop.Resource.fromLocalFile('template.docx'),
            config,
        );

        // Execute the print job and save the response to a list
        outputFilesProm.push(printJob.execute());
    }

    const outputFiles = await Promise.all(outputFilesProm);

    // Wait for the buffers of the server responses
    const buffersProm = outputFiles.map((res) => res.buffer);
    const buffers = await Promise.all(buffersProm);
})();

(async () => {
    const outputFilesProm: Promise<cop.Response>[] = [];

    for (let i = 0; i < 10; i += 1) {
        // Create print job with 10 data elements
        const dataSplit: { [key: string]: cop.elements.Element } = {};
        Object.entries(data)
            .slice(i * 10, (i + 1) * 10)
            .forEach(([key, value]) => {
                dataSplit[key] = value;
            });
        const printJob = new cop.PrintJob(
            dataSplit,
            server,
            cop.Resource.fromLocalFile('template.docx'),
            config,
        );

        // Execute the print job and save the response to a list
        outputFilesProm.push(printJob.execute());
    }

    const outputFiles = await Promise.all(outputFilesProm);

    // Wait for the buffers of the server responses
    const buffersProm: Promise<ArrayBuffer>[] = outputFiles.map(
        (res) => res.buffer,
    );
    const buffers = await Promise.all(buffersProm);
})();

// Load template
Base64Resource template = new Base64Resource();
InputStream resourceAsStream = getClass().getResourceAsStream("template.docx");
byte[] targetArray = new byte[resourceAsStream.available()];
resourceAsStream.read(targetArray);
String encodedString = Base64.getEncoder().encodeToString(targetArray);
template.setFileBase64(encodedString);
template.setFiletype("docx");
template.setMimeType(Mimetype.getMimeType("docx"));

Response[] outputFiles = new Response[10];
Set<Map.Entry<String, RenderElement>> dataEntries = data.entrySet();
Iterable<List<Map.Entry<String, RenderElement>>> slicedDataEntries = Iterables.partition(dataEntries, 10);
int index = 0;
for (List<Map.Entry<String, RenderElement>> i : slicedDataEntries) {
    Hashtable<String, RenderElement> d = new Hashtable<String, RenderElement>();
    for (Map.Entry<String, RenderElement> el : i) {
        d.put(el.getKey(), el.getValue());
    }
    PrintJob printjob = new PrintJob(d, copServer, conf, template, null, null, null, null);
    outputFiles[index] = printjob.execute();
    index++;
}

Now that we saved the server response for all the smaller tasks, we create the final request to merge all the received (merged) PDFs. Therefore we create Resource-objects from the Response-objects.

resources = [
    cop.Resource.from_raw(raw_data=response.binary, filetype="pdf")
    for response in output_files
]

const resources = buffers.map((buff) =>
    cop.Resource.fromRaw(Buffer.from(buff), 'pdf'),
);

const resources: cop.resource.RawResource[] = buffers.map((buff) =>
    cop.Resource.fromRaw(Buffer.from(buff), 'pdf'),
);

// Create Resource-objects from the Response-objects in output_files
Base64Resource[] resources = new Base64Resource[outputFiles.length];
for (int i = 0; i < outputFiles.length; i++) {
    String resourceBase64String = new String(Base64.getEncoder().encode(outputFiles[i].getBody()));
    resources[i] = new Base64Resource();
    resources[i].setFileBase64(resourceBase64String);
    resources[i].setFiletype("pdf");
    resources[i].setMimeType(Mimetype.getMimeType("pdf"));
}

Finally, we create the print job for the last request that merges the 10 merged PDF's. As the template we pick the first PDF in the resources-list and the other 9 PDFs from the resources-list can be added as files that need to be appended to the template file.

print_job = cop.PrintJob(
    data=cop.elements.Property("not_used", "not_used"),
    server=server,
    template=resources[0],
    append_files=resources[1:],
)

response = print_job.execute()
response.to_file("output")

(async () => {
    const printJob = new cop.PrintJob(
        new cop.elements.Property('not_used', 'not_used'),
        server,
        resources[0],
        undefined,
        undefined,
        undefined,
        resources.slice(1, resources.length),
    );

    const response = await printJob.execute();
    await response.toFile('output');
})();

(async () => {
    const printJob = new cop.PrintJob(
        new cop.elements.Property('not_used', 'not_used'),
        server,
        resources[0],
        undefined,
        undefined,
        undefined,
        resources.slice(1, resources.length),
    );

    const response = await printJob.execute();
    await response.toFile('output');
})();

// Create the print job for the last request that merges the 10 merged PDF's
// As the template we pick the first PDF in the resources-list
// The other 9 PDFs from the resources-list can be added to append_files (or
// prepend_files)
Base64Resource[] splitResources = Arrays.copyOfRange(resources, 1, resources.length);
data = new Hashtable<String, RenderElement>();
data.put("not_used", new Property("not", "used"));
Output conf2 = new Output("pdf", "raw", "libreoffice", null, null, null, null);
PrintJob printjob = new PrintJob(data, copServer, conf2, resources[0], null, null, splitResources, null);

printjob.execute().downloadLocally("./downloads/multiple_request_merge");

More Info

For more information about the multiple request merge example for each individual SDK, visit the appropriate pages:

• Python (Code)
• JavaScript (Code)
• Java (Code)