Bulk insert into the SQL Server 2012 database - C#

user3516668

I have about a million of records in the list. And foreach in the list I query the database to get the modified date based on id. Once I get it, it's test in the condition to either insert it or update it .. but is there any way to do it as bulk (at once or in batches) insertion or update..??

foreach (No_kemi no_list in newforSQL)
{
    DateTime eModifiedDate = no_list.ModifiedDate.Value;
    string Id = no_list.ID.ToString();

    String selectQuery = "SELECT ModifiedDate FROM NO_table WHERE ID = '" + Id + "'";
    DateTime? dbmoddate = null;

    using (SqlCommand selectCommand = new SqlCommand(selectQuery, connection))
    {
        // Use the above SqlCommand object to create a SqlDataReader object.
        using (SqlDataReader rdr = selectCommand.ExecuteReader())
        {
            while (rdr.Read())
            {
                dbmoddate = (DateTime)rdr["ModifiedDate"];
            }
        }
    }

    string eType = null;
    string SubGUID = null;
    string trimSubKey = no_list.SubscriberKey;

    try
    {
        if ((trimSubKey.Length > 3) && (trimSubKey != ""))
        {
            if (trimSubKey.Contains(","))
            {
                string[] values = trimSubKey.Split(',');

                SubGUID = values[0];
                eType = values[1];
            }
            else 
            { 
                SubGUID = trimSubKey; 
                eType = ""; 
            }
        }
        else 
        { 
             SubGUID = ""; 
             eType = ""; 
        }
    }
    catch (Exception ex)
    {
        logger.Error("Error : " + ex.Message );
    }

    if (dbmoddate == null)
    {
        DateTime no_listinsetdate = DateTime.Now;
        String insertQuery = "INSERT INTO NO_table (CreatedDate, ModifiedDate, ID, eType, SubGUID, DbDate) VALUES (@CreatedDate, @ModifiedDate, @ID, @eType, @SubGUID, @DbDate,)";

        using (SqlCommand insertCommand = new SqlCommand(insertQuery, connection))
        {
             insertCommand.Parameters.AddWithValue("@CreatedDate", no_list.CreatedDate);
             insertCommand.Parameters.AddWithValue("@ModifiedDate", no_list.ModifiedDate);
             insertCommand.Parameters.AddWithValue("@ID", no_list.ID);
             insertCommand.Parameters.AddWithValue("@eType", eType);
             insertCommand.Parameters.AddWithValue("@SubGUID", SubGUID);
             insertCommand.Parameters.AddWithValue("@DbDate", no_listinsetdate);

             insertCommand.ExecuteNonQuery(); // execute the query
         }
    }
    else if (eModifiedDate > dbmoddate)
    {
        DateTime no_listinsetdate = DateTime.Now;
        String updateQuery = "UPDATE  NO_table SET CreatedDate = @CreatedDate, ModifiedDate = @ModifiedDate, ID = @ID, eType = @eType, SubGUID = @SubGUID, DbDate = @DbDate WHERE ID = '" + Id + "'";

        using (SqlCommand updateCommand = new SqlCommand(updateQuery, connection))
        {
            updateCommand.Parameters.AddWithValue("@CreatedDate", no_list.CreatedDate);
            updateCommand.Parameters.AddWithValue("@ModifiedDate", no_list.ModifiedDate);
            updateCommand.Parameters.AddWithValue("@ID", no_list.ID);
            updateCommand.Parameters.AddWithValue("@eType", eType);
            updateCommand.Parameters.AddWithValue("@SubGUID", SubGUID);
            updateCommand.Parameters.AddWithValue("@DbDate", no_listinsetdate);

            updateCommand.ExecuteNonQuery(); // execute the query
        }
    }
}

any way to speed up the process... it takes lot of time to finish the operations... or some time I get timeout error ...

thx

nvoigt

This is way to broad to actually have a coded solution here but you should be able to create one yourself based on general advice:

You could probably save on a lot of network roundtrips if you wrote a SQL procedure that does this. Further optimization would result in this SQL procedure taking batches of records instead of a single one so you can for example send 1000 records, the procedure will loop over them and for each do the query and resulting update/insert. That means the results do not have to go back and forth across the network, resulting in less traffic and shorter runtime.

Maybe your actions can be postulated as a single MERGE command instead of query and update/insert. This depends on your table structure, data and intention.

Terms to google for: stored procedure, table type, MERGE.

この記事はインターネットから収集されたものであり、転載の際にはソースを示してください。

侵害の場合は、連絡してください[email protected]

編集
0

コメントを追加

0

関連記事

分類Dev

Move database from SQL Server 2012 to 2008

分類Dev

SQL Server Bulk Insert Format FIle could not be opened

分類Dev

Rollback in single stored procedure for forloop & insert query in SQL Server 2012

分類Dev

SQL Server 2012 Insert Stored Procedure for N parameters

分類Dev

How to search a column name in all tables in a database in SQL Server 2012?

分類Dev

Get row count of successful bulk insert using jdbc from sql server

分類Dev

Unexpected characters on bulk insert with t-sql

分類Dev

Bulk Insert with Linq to Sql (vb.net)

分類Dev

Export SQL Server 2014 database (2008 compatible) in a backup file compatible with 2012

分類Dev

Numbering islands in SQL Server 2012

分類Dev

SQL Server 2012 Pivot Table

分類Dev

Restore SQL Server 2012 error

分類Dev

Pandas insert into SQL Server

分類Dev

Insert data in Sql Server datable from Datagridview c#

分類Dev

Exception while excecuting insert query to SQL Server from c#

分類Dev

Insert and update a datetime into SQL database

分類Dev

How to Insert value to sql server database from data table vb.net

分類Dev

Edit SQL Server Database (INSERT, UPDATE, etc) in ASP.NET (VB)

分類Dev

No process is on the other end of the pipe (SQL Server 2012)

分類Dev

SQL Server2012のRANDBETWEEN

分類Dev

Requires SQL Server 2012 Express LocalDB

分類Dev

SQL Server 2012のJSON_VALUE?

分類Dev

How to check history in SQL Server 2012?

分類Dev

SQL Server 2012 Pivot Dynamic with Values?

分類Dev

SQL Server2014とSQLServer 2012

分類Dev

Merge data into one column - sql server 2012

分類Dev

authentication with node.js in SQL server 2012

分類Dev

PHP Connection to MS SQL Server 2012

分類Dev

MS-SQl Server 2012 for ubuntu

Related 関連記事

ホットタグ

アーカイブ