Bypass Salesforce Governor Limits when Working with Files

Introduction

Salesforce is a multi-tenant system. This means Salesforce resources are shared by all Salesforce accounts. To prevent the system from abuse and to maintain good performance across all Salesforce accounts, Salesforce applies various limits on accounts & transactions. These are called governor limits.

Salesforce mentions all the governor limits here. Although there are various limits, only some of them become restrictive when working with files in apex.

In this article, we explore what are the important file-related governor limits, why they can become restrictive and finally how to bypass them using a little code in apex (or outside of it).

File Use-cases & Governor Limits

Files are an essential part of the sales process. As a Salesforce developer, you would often come across use cases that involve working with files, mainly uploading files from Salesforce to cloud drives such as Sharepoint, Google Drive, OneDrive, Box, Dropbox, AWS S3 etc…

Here are some common examples of when such file uploads might be needed when working in apex:

1. When someone uploads a file into chatter feed or the notes and attachments section
2. When a file comes in as an email attachment
3. When community users upload files from their portals
4. When sales/service teams need to upload files from a lightning web component

Don't forget to check out: Batch Apex in Salesforce (Basics, Governor Limits, Custom Iterable of Batch)

File-related Governor Limits

Salesforce has a bunch of standard objects that store binary data (blob) and are used for file management.

There are 6 main objects that are used to meet all the file management needs. These are Attachment, ContentDocument, ContentNote, Document, Folder & Note. The "Files" Object that is commonly used by sales & service cloud users uses the ContentDocument object.

When you work with these objects in apex, you would generally run into governor limits like heap size & callout timeouts. Here are some common governor limits you would run into -

1. Maximum heap size - It is the amount of memory that is used by an apex transaction. For synchronous methods, it is 6 MB and for asynchronous methods, it is 12 MB. This means, that if you read a file as a blob in apex, you can only process the files with sizes less than these limits
2. Maximum cumulative timeout for all callouts - It is the time that all HTTP calls would take to finish an apex transaction. This limit is 120 seconds. This means that if you’re uploading/downloading the file to/from an external service, it needs to finish within 2 minutes
3. Maximum execution time for each Apex transaction -  This limit is 10 minutes for both synchronous & asynchronous apex methods. If you’re doing long-running tasks like converting a file to pdf, it needs to finish within 10 minutes

How to bypass the Governor Limits

One way to work around the governor limits while uploading files to external storage from Salesforce is by using the SObject Blob Retrieve API. This is a REST API which gives binary content of any Salesforce object that has a blob field.

This method requires an external service that makes an authenticated call to the blob retrieve API to fetch the binary data and put it on external storage. Since REST API doesn’t impose a response size limit & timeouts, this successfully works around the file size limit.

Here is an example of how to upload a pdf attachment (ContentDocument in lightning) to AWS S3 using javascript.

import axios from 'axios';
import aws from 'aws-sdk';
const { data } = await axios.get(
    'https://ap5.salesforce.com/services/data/v55.0/sobjects/ContentVersion/0696D000001Pmk5QAC/VersionData',{
        headers: {
            Authorization: `Bearer ${salesforceAccessToken}`
        },
    }
);
const s3 = new aws.S3({
    accessKeyId: AWS_ACCESS_KEY_ID,
    secretAccessKey: process.env.AWS_ACCESS_KEY_SECRET,
});
s3.upload({
    Bucket: 'UPLOAD_BUCKET_NAME',
    Key: 'test.pdf',
    Body: data,
    ContentType: 'application/pdf',
});

One catch with this approach is that this code needs to run outside of Salesforce using an API server or cloud functions like Salesforce functions, AWS lambda or Azure functions.

Uploading to Other Cloud Storage Providers

The same technique can be used to upload the files to Google Drive, OneDrive, Sharepoint, Box, Dropbox using the appropriate upload API and supplying the right authentication.

We recommend using OAuth2 for authentication

Thus a simple combination of a cloud function and an authentication mechanism can help you bypass governor limits and transfer any type of files between Salesforce storage.

There are 3rd party tools that make this extremely easy.

Check out an amazing Salesforce video tutorial here: Governor Limits in Salesforce | Video Tutorial

Upload Using a 3rd Party Tool

You can use 3rd party tools from AppExchange that make working with files easy. One such tool is CloudFiles. Using CloudFiles, you can avoid setting up an API server, setting up authentications or building integrations with different cloud storage platforms.

The same functionality described above can be achieved using CloudFiles with one line of Apex code:

cldfs.Client.upload(
    files, // a list of sobjects with blob field
    'rTmK2YQnxaEIFkU_C18a62b2xWyDB', // folder id in external storage
    'google', // destination: google|onedrive|sharepoint|box|dropbox|cloudfiles(s3)
    'mydrive', // drive id in external storage(optional)
    null // site id in case of sharepoint(optional for other storages)
);

Users can authenticate Salesforce & destination file storage using the CloudFiles UI. It’s one time, one-click setup process.

You can use the upload code as part of flows or apex triggers to automate file management in Salesforce. All this without ever hitting the governor limits!

Happy coding!