Categories

Processing Large Data Sets in Laravel with Lazy Collections and Cursors

Donate with

82uymVXLkvVbB4c4JpTd1tYm1yj1cKPKR2wqmw3XF8YXKTmY7JrTriP4pVwp2EJYBnCFdXhLq4zfFA6ic7VAWCFX5wfQbCC

Copy Monero Donation Address

Donate with

Donate BAT now

Verified BAT publisher

Introduction

When working with thousands or millions of records in a Laravel application, memory usage can quickly become a bottleneck. Traditional methods like all() or get() eagerly load all data into memory, which can be inefficient or even crash your application.

Laravel’s Lazy Collections, introduced in Laravel 6, offer a smarter way to process large datasets efficiently. Combined with Eloquent’s cursor() method, you can iterate over massive data sets without overwhelming memory.

In this guide, we'll explore how Lazy Collections work, when to use them, and how to integrate them with Eloquent and database queries.

What Is a Lazy Collection?

Lazy Collections defer computation until it's needed, using generators under the hood. Unlike traditional collections, Lazy Collections don’t hold everything in memory.

use Illuminate\Support\LazyCollection;

LazyCollection::make(function () {
    for (\$i = 0; \$i < 1000000; \$i++) { yield \$i; } })->take(10)->each(function (\$value) {
    dump(\$value);
});

Even though we define a million elements, only 10 are processed, making it extremely efficient.

When Should You Use Lazy Collections?

• Processing large tables in background jobs or Artisan commands
• Exporting millions of rows to CSV
• Streaming data without exhausting memory
• Integrating with APIs or logs that produce unbounded streams

Using Eloquent with Lazy Collections: cursor()

The cursor() method returns a Lazy Collection backed by a database cursor:

User::cursor()->each(function (\$user) {
    // Only one row is loaded into memory at a time
    echo \$user->email . "\n";
});

Compare this with get():

User::get()->each(...); // 🚨 Loads all records into memory at once

With cursor(), Laravel fetches one row at a time and keeps memory usage stable.

Use Case: Exporting Users to CSV

Here’s an example that exports millions of users efficiently:

use Illuminate\Support\Facades\Storage;

Storage::put('users.csv', '');

User::cursor()->each(function (\$user) {
    Storage::append('users.csv', "{\$user->id}, {\$user->email}");
});

This approach prevents memory bloat and scales linearly.

Combining with LazyCollection Methods

You can use all LazyCollection methods:

$filtered = User::cursor()
    ->filter(fn(\$user) => \$user->is_active)
    ->map(fn(\$user) => strtoupper(\$user->email))
    ->take(1000);

foreach (\$filtered as \$email) {
    echo \$email . "\n";
}

Best Practices and Caveats

Don’t use ->count() or ->sum() on a LazyCollection unless you exhaust it (it will iterate all items).

Avoid N+1 queries — still eager load relations when needed:

User::with('roles')->cursor()->each(...);

If sorting is needed, ensure indexes exist on your database columns.

Conclusion

Lazy Collections are a powerful tool in Laravel for handling large datasets. By using cursor() and LazyCollection methods, you can create scalable, memory-safe workflows — whether you’re exporting data, running background jobs, or building CLI tools.

If your app is dealing with large tables, this is a feature you can’t afford to ignore.