scout:queue-import: Faster Indexing in Laravel Scout

• Speed Up Laravel Scout Indexing with scout:queue-import
• The Problem with scout:import
• The Solution: scout:queue-import
• Usage Example - php artisan scout:queue-import
• Important Notes
• Why It Matters

Speed Up Laravel Scout Indexing with scout:queue-import

If you've ever waited hours or even days for php artisan scout:import to finish indexing your models, you're not alone. A new command coming to Laravel Scout aims to solve exactly that: scout:queue-import.

The new Artisan command contributed by @ste_bau was introduced in PR #929, and it's designed to make importing millions of records into your search indexes dramatically faster by leveraging your queue system more efficiently.

The Problem with scout:import

The existing scout:import command loops through your entire table, loading and then indexing chunks of your models sequentially. While this works fine for smaller datasets, it's painfully slow for large applications. Even if you're using queues, jobs will be dispatched one at a time only after loading the next chunk.

The Solution: scout:queue-import

The new scout:queue-import command changes the approach entirely. Instead of iterating through the table, it:

1. Retrieves the MIN and MAX values of your model’s Scout key (usually the id).
2. Splits that range into chunks (default: 500), generating multiple ranges.
3. Dispatches a job per range to your queue workers.

Because it avoids querying the entire database to fetch rows before queuing, this method is much more efficient especially when your queue workers can run in parallel. With multiple queue workers running, jobs can be processed in parallel something that's not possible with the sequential nature of the scout:import command.

Usage Example - php artisan scout:queue-import

php artisan scout:queue-import "App\\Models\\Post"

You can also customize the chunk size:

php artisan scout:queue-import "App\\Models\\Post" --chunk=1000

You can also customize the range:

php artisan scout:queue-import "App\Models\Post" --min=10 --max=100

Important Notes

• Your model’s Scout key must be numeric and auto-incrementing. If it isn’t, the command will throw an error and abort.
• Gaps in the ID sequence may cause jobs to index fewer records (or none at all) in that range.
• You must have queue workers running and properly configured for the jobs to be processed.

Why It Matters

If you're dealing with large datasets, scout:queue-import can be a game changer. Instead of waiting hours or days, you can reindex millions of records quickly without putting your system on pause.

Give it a try and speed up your search indexing workflow with Laravel Scout!