Problems encountered in inserting large quantities of data into the database in the project

Last Friday, I ran a lot of data in the test environment ( There are about 8000 ten thousand ), I want to see it on Monday , But it was found on Monday that the data had not been processed , Stop , Problems found while checking logs , The first is to see “no space left on device” This anomaly , Oh , There's not enough disk space , Delete it …,df- I Check the usage of file attributes :

Very few are used , Then use df -h Look at the actual file size :

( This picture is deleted , I forgot the screenshot ……)

The use of the place marked with red has reached 100G, Reached 100%; So obviously , The server is out of memory ; Then you can find some useless log files and delete them ;

When I continue to run the data , After a while, I found that the data did not move , So I went to check the log , Another new anomaly is found :” ORA-01653: unable to extend table Table name by 1024 in tablespace Table space name ”

Ha , Insufficient daily table space ;

First check oracle Table space ：

SELECT UPPER(F.TABLESPACE_NAME) " Table space name ",

D.TOT_GROOTTE_MB " Table space size (M)",

D.TOT_GROOTTE_MB - F.TOTAL_BYTES " Used space (M)",

TO_CHAR(ROUND((D.TOT_GROOTTE_MB - F.TOTAL_BYTES) / D.TOT_GROOTTE_MB * 100,2),'990.99') || '%' " Use proportion ",

F.TOTAL_BYTES " Free space (M)",

F.MAX_BYTES " The largest space (M)"

FROM (SELECT TABLESPACE_NAME,

ROUND(SUM(BYTES) / (1024 * 1024), 2) TOTAL_BYTES,

ROUND(MAX(BYTES) / (1024 * 1024), 2) MAX_BYTES

FROM SYS.DBA_FREE_SPACE

GROUP BY TABLESPACE_NAME) F,

(SELECT DD.TABLESPACE_NAME,

ROUND(SUM(DD.BYTES) / (1024 * 1024), 2) TOT_GROOTTE_MB

FROM SYS.DBA_DATA_FILES DD

GROUP BY DD.TABLESPACE_NAME) D

WHERE D.TABLESPACE_NAME = F.TABLESPACE_NAME

ORDER BY 4 desc ;

Query results ( This picture is expanded , It used to be 60G, I forgot the screenshot ……)

So at that time, the table space was insufficient , Need to expand table space ;

View the data files used by the tablespace ：

select file_name,tablespace_name,bytes/1024/1024,maxbytes/1024/1024 from dba_data_files where tablespace_name=' Table space name ';

(4 It is added after expansion , It turned out that there was only 60 individual G,)

Expand at most each time 32G,( As for how to expand , I don't know , Administrator extended ,)

Query the most occupied table space 10 Objects :

select OWNER,SEGMENT_NAME,SEGMENT_TYPE,total||'M' from

(select OWNER,SEGMENT_NAME,SEGMENT_TYPE,bytes/1024/1024 total from dba_segments

where TABLESPACE_NAME=' Table space name ' order by bytes/1024/1024 desc ) where rownum <11;

The first two columns are tablespace name and table name , I don't want to show it , These two columns are type and occupancy size ;

The table space is expanded , The server memory is enough , This time we can continue to run the data , however ………… It didn't take long to collapse again !!!

I wonder if you have seen , Anyway, it's my first time to see , After consulting some materials, I learned :

ORA-00604： Recurse a SQL Error in layer

ORA-01000: Exceeded the maximum number of open cursors

Because of my business needs , When processing data , Circular insertion is required , I am every 500 Access the database once , this 500 If there is one, update it. If not, insert it ;

So these combine to mean , Because of my sql Has been inserting data circularly , The cursor is always on , and oracle The default number of cursors is 300, And I've been cycling , Instead of closing , It broke down ;

I found... On the Internet 2 Kind of solution :

The first is to increase oracle The maximum number of cursors ;( But it is said that this indicator does not cure the root cause );

The second is to use it in the loop of the code JDBC Close it manually , release ( You can check the relevant information online )

But I listened to the advice of a great God , Just put 500 Modified to 50, Just fine ……!!!!!!!

I'm just here to record the experience of this wonderful race , Don't touch the gods if they don't like it ~