Deleting billions of records from Oracle Table containing trillions of records

Mayank Tripathi

I need to delete approx. of 30 billion of rows from an Oracle table that has about 100 + billion of rows. I have all the IDs of the rows I need to delete in a temporary table. Currently I am using single delete statement as below, also using SUBPARTITION and created Index on the temp_table. However this took 4+ hrs to complete in PRODUCTION.

DELETE FROM table_name SUBPARTITION(subpartition_name) WHERE id IN (SELECT id FROM temp_table);
COMMIT;

Is there a way I can optimize this to run bit fast.

Just for a note :

  1. The oracle table I am referring is common for multiple clients, so the below option is not suitable here. Creating new table and move the required data into it and drop the old table followed by renaming the new table to old table.
  2. Delete in Batch : Looping over the temp table and deleting something like below, is taking more time in non-prod environment, and not sure how it goes in production env.
DECLARE
  vCT NUMBER(38) := 0;

BEGIN
  FOR t IN (SELECT id FROM temp_table) LOOP
    DELETE FROM table_name WHERE id = t.id;
    ......
    ......
      COMMIT;
    END IF;
  END LOOP;
  COMMIT;
END;
  1. Option to create individual DELETE statement is also not feasible here as the record count is in Billions.

I did checked, there are partitions and sub-partitions on the table which I am utilizing it, and there is no child table dependent on it.

Please suggest.

Marmite Bomber

Delete (or update) large number of rows is pain and take lot of time.

The most effective way to manage it is to use the updateble join views

What is requested, but shold be no problem is a unique index on your delta table containing the id to be deleted.

Also while using this approach be sure, that the large table has no unique index on the id column. There are some technical issues if both tables have unique index on the join column.

Than use following query (if you want to use sub-partition wise) delete

delete from 
(
select delta.id, big.id big_id 
from delta 
join big subpartition (SYS_SUBP220880)
on delta.id = big.id  
)

Oracle uses the hash join of both tables which is in your case the only possibility to manage your scale. You may dedploy parallel option, do not forget to enable it.

This is the expected execution plan:

-------------------------------------------------------------------------------------------------------
| Id  | Operation                     | Name  | Rows  | Bytes | Cost (%CPU)| Time     | Pstart| Pstop |
-------------------------------------------------------------------------------------------------------
|   0 | DELETE STATEMENT              |       |   100K|  1757K|   875   (2)| 00:00:01 |       |       |
|   1 |  DELETE                       | BIG   |       |       |            |          |       |       |
|*  2 |   HASH JOIN                   |       |   100K|  1757K|   875   (2)| 00:00:01 |       |       |
|   3 |    TABLE ACCESS FULL          | DELTA |   100K|   488K|    47   (3)| 00:00:01 |       |       |
|   4 |    PARTITION COMBINED ITERATOR|       |   783K|  9947K|   825   (1)| 00:00:01 |   KEY |   KEY |
|   5 |     TABLE ACCESS FULL         | BIG   |   783K|  9947K|   825   (1)| 00:00:01 |    65 |    65 |
-------------------------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------
 
   2 - access("DELTA"."ID"="BIG"."ID")

Collected from the Internet

Please contact [email protected] to delete if infringement.

edited at
0

Comments

0 comments
Login to comment

Related

From Dev

Deleting billions of records from Oracle Table containing trillions of records

From Dev

Mysql: 7 billions records in a table

From Dev

Move billions of records from one table to another in teradata

From Dev

deleting records from table using SQLAlchemy

From Dev

deleting records from a table where there is an inner join

From Dev

All records being deleted when deleting records from a temp table

From Dev

Deleting large records in oracle sql

From Dev

How to delete duplicate records from a table in oracle

From Dev

Preventing users from deleting records in one table that have related records in another table

From Dev

Finding duplicate records from table and deleting all but one with latest date

From Dev

Deleting all table records from core data database using relationship

From Dev

Deleting checked records from table in asp.net

From Dev

Finding duplicate records from table and deleting all but one with latest date

From Dev

Will a many-to-many table with billions of records cause performance issue?

From Dev

Select from a table based on another table Number of records Oracle

From Dev

Deleting multiple records in a table using join in Peewee?

From Dev

Deleting Records in SQL server table prior to insert

From Dev

Extract records from table

From Dev

Returning a table of records from an Oracle function into a C# application

From Dev

Oracle SQL query: Delete the oldest duplicate records from the table

From Dev

Oracle Query - Selecting records using column data from first table

From Dev

How to retrieve half of records from a table - Oracle 11g

From Dev

How to import millions of records from flat file to oracle table

From Dev

inserting records from two different tables into a single table in oracle

From Dev

get records without records from another table

From Dev

C++ : Deleting records from binary file

From Dev

Deleting multiple records from sql bound Datagrid

From Dev

Problems with deleting old records from MySQL

From Dev

deleting records from sqlite on the basis of date in android

Related Related

  1. 1

    Deleting billions of records from Oracle Table containing trillions of records

  2. 2

    Mysql: 7 billions records in a table

  3. 3

    Move billions of records from one table to another in teradata

  4. 4

    deleting records from table using SQLAlchemy

  5. 5

    deleting records from a table where there is an inner join

  6. 6

    All records being deleted when deleting records from a temp table

  7. 7

    Deleting large records in oracle sql

  8. 8

    How to delete duplicate records from a table in oracle

  9. 9

    Preventing users from deleting records in one table that have related records in another table

  10. 10

    Finding duplicate records from table and deleting all but one with latest date

  11. 11

    Deleting all table records from core data database using relationship

  12. 12

    Deleting checked records from table in asp.net

  13. 13

    Finding duplicate records from table and deleting all but one with latest date

  14. 14

    Will a many-to-many table with billions of records cause performance issue?

  15. 15

    Select from a table based on another table Number of records Oracle

  16. 16

    Deleting multiple records in a table using join in Peewee?

  17. 17

    Deleting Records in SQL server table prior to insert

  18. 18

    Extract records from table

  19. 19

    Returning a table of records from an Oracle function into a C# application

  20. 20

    Oracle SQL query: Delete the oldest duplicate records from the table

  21. 21

    Oracle Query - Selecting records using column data from first table

  22. 22

    How to retrieve half of records from a table - Oracle 11g

  23. 23

    How to import millions of records from flat file to oracle table

  24. 24

    inserting records from two different tables into a single table in oracle

  25. 25

    get records without records from another table

  26. 26

    C++ : Deleting records from binary file

  27. 27

    Deleting multiple records from sql bound Datagrid

  28. 28

    Problems with deleting old records from MySQL

  29. 29

    deleting records from sqlite on the basis of date in android

HotTag

Archive