I am pulling out large data from oracle database using cx_oracle using below sample script:
from cx_Oracle import connect
TABLEDATA = []
con = connect("user/password@host")
curs = con.cursor()
curs.execute("select * from TABLE where rownum < 100000")
for row in curs:
TABLEDATA.append([str(col) for col in list(row)])
curs.close()
con.close()
Problem with storing in list is that it ends up to about 800-900 mb of RAM usages. I know I can instead save this in file and not store in list but I am using this list to display table using QTABLEVIEW and QABSTRACTTABLE MODEL.
Is there any alternate or more effient way where I can minimise memory usage of storing this data and also use it to display my table?
I have tried multiple possobilities, I don't think qsqltablemodel works for me. Though it load data directly from database, as you keep scrolling down it loads more and more data in table and hence the memory usage keep on increasing.
What I think will ideally work is being able to load set number of rows in model. As you scroll down it loads new rows but also at the same time unloads what's already there. So at any point of time we only have set number of rows loaded in model.
Collected from the Internet
Please contact [email protected] to delete if infringement.
Comments