Skip to content
javahacks.dev
Hibernate

Hibernate Sequence Batch Insert With Safe ID Generators

Speed up bulk inserts in Hibernate with sequence ids, jdbc.batch_size, ordered inserts and periodic flush and clear, then check the batches ran.

6 min read
Summarize in AI
Flat illustration of an entity class mapped to a database table, with the Hibernate, OpenJDK, Liquibase, MySQL and MariaDB logos.

Inserting 100,000 rows one statement at a time spends most of its time on network round trips. JDBC batching sends many statements in one round trip. This is on Hibernate 7 and PostgreSQL 17.

Step 1: ids that allow batching

java
@Id
@GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "item_seq")
@SequenceGenerator(name = "item_seq", sequenceName = "item_seq", allocationSize = 50)
private Long id;
SQL
create sequence item_seq start with 1 increment by 50;

With IDENTITY, Hibernate has to run each insert immediately to learn the id, so nothing can be queued. The comparison is in id generation strategies.

Step 2: configure batching

application.propertiesproperties
spring.jpa.properties.hibernate.jdbc.batch_size=50
spring.jpa.properties.hibernate.order_inserts=true
spring.jpa.properties.hibernate.order_updates=true
spring.datasource.url=jdbc:postgresql://localhost:5432/shop?reWriteBatchedInserts=true

batch_size is the maximum statements per batch. order_inserts sorts pending inserts by entity type, so inserts into item are not interleaved with inserts into item_tag, which would split the batch. reWriteBatchedInserts lets the PostgreSQL driver turn a batch into a multi-row insert. The JDBC side of batching is in JDBC batch updates.

Step 3: flush and clear

java
@Transactional
public void importItems(List<ItemRow> rows) {
    int count = 0;
    for (ItemRow row : rows) {
        entityManager.persist(new Item(row.name(), row.price()));
        if (++count % 50 == 0) {
            entityManager.flush();     // send the batch
            entityManager.clear();     // detach, free memory
        }
    }
}

The persistence context keeps a reference to every managed entity until it is cleared. Without clear(), 100,000 inserts hold 100,000 objects and a snapshot of each, which ends in Java heap space. flush() at the same boundary makes the batch visible as a unit.

Step 4: check that it batched

Hibernate can count its own batches:

properties
spring.jpa.properties.hibernate.generate_statistics=true
logging.level.org.hibernate.engine.jdbc.batch.internal.BatchingBatch=DEBUG

The statistics report JDBC batches executed and JDBC statements. If the statements equal the batches, batching is not happening. A datasource proxy library prints actual batch sizes. The usual causes of no batching are IDENTITY, a missing batch_size, mixed entity types without order_inserts, and saveAll with entities that already have ids, which triggers a select-then-merge per row.

Updates and versioned entities

order_updates applies the same idea to updates. Entities with @Version need batch versioning, which Hibernate enables by default (hibernate.jdbc.batch_versioned_data), and some older drivers misreport the row count. If you see an optimistic lock failure only under batching, see StaleObjectStateException.

When to skip the ORM

For a few hundred thousand rows with no behaviour, JdbcTemplate.batchUpdate or the database's bulk loader (COPY on PostgreSQL) is faster still, since there are no entities to track. Hibernate batching is a good fit when the objects already exist for other reasons.

Why identity columns defeat batching

With an identity column, the database assigns the id when the row is inserted, so Hibernate must execute each insert immediately to learn the id, one statement per entity, and cannot group them. A sequence is different: Hibernate can ask the database for ids ahead of time, and then send many inserts together because it already knows every key. That is why the generator is the first thing to look at when an import is slow, and why a table created with GENERATED ALWAYS AS IDENTITY and the default strategy gives you no batching at all, whatever you set in the properties.

The allocation size mismatch

A sequence-based generator has an allocationSize, the number of ids Hibernate reserves per round trip, and the sequence in the database must increment by the same number. If the entity says 50 and the sequence increments by 1, Hibernate generates ids that collide with the rows another instance wrote, and you see duplicate key errors that appear only under concurrency. Keep both at the same value, commonly 50, and check the sequence definition in the migration. A mismatch is the most common cause of mysterious key violations after someone enables a sequence.

Memory and the persistence context

Every entity you persist stays in the persistence context until you clear it, and a loop of a million inserts holds a million objects, plus the dirty-checking snapshot of each. The heap fills and the process slows long before it fails. Flush and clear every batch of a few hundred rows, as in the loop above, and the memory stays flat. Remember that clearing detaches everything, so do not use an entity from before the clear afterwards, and re-fetch what you need by id.

Ordering inserts and updates

When entities of several types are inserted in one transaction, Hibernate may interleave the statements, which breaks batches into many small ones. Set hibernate.order_inserts=true and hibernate.order_updates=true, and it groups statements by table first. The setting costs a little sorting and often doubles the effective batch size for object graphs. Combined with the driver flag that rewrites batches, covered in JDBC batching, the number of round trips falls by an order of magnitude.

Verifying with a counter

Do not trust the configuration, and count. Wrap the data source in a statement-counting proxy in a test, insert a thousand entities, and assert that the number of statements is about twenty and not a thousand. The test takes a few lines, and it catches the day someone changes the id generator or removes the property while tidying the configuration. Without it, the first symptom is a slow import in production.

When Hibernate is the wrong tool

For tens of millions of rows, an ORM is overhead you do not need. Use JDBC batches, a bulk loader such as PostgreSQL COPY, or a staging table loaded by the database's own import and merged with one SQL statement. Keep Hibernate for the business logic around the data, where its features pay off.

Cost of the sequence round trip

Hibernate fetches a block of ids with one call to the sequence per allocationSize entities, so with 50 it makes one call per 50 inserts. That is cheap, and the gap in ids after a restart is harmless but visible, so do not let anything depend on ids being contiguous. Reports that count by the maximum id, or code that assumes the next id is the last plus one, are wrong with any pooled generator.

Large imports in practice

For a real import job, read the source in chunks, one transaction per chunk of a few thousand rows, and log progress after each. If a chunk fails, the earlier ones are safe, and the job can resume from the last committed position instead of starting over. Record the position in a table, so a restart after a crash skips what is done. A single transaction around a million inserts holds locks and undo space for the whole duration, and a failure at row 999,999 throws away an hour of work.

Where this does not apply

On MySQL, rewriteBatchedStatements=true on the URL plays the part that reWriteBatchedInserts plays on PostgreSQL. Batching does not help a transaction that inserts three rows. The reference is Batching, and the PostgreSQL driver docs list the URL options.

Share

Keep reading

Get new posts by email

We send one email when a new post is published. No spam.

Unsubscribe in one click. No spam, ever.