How to query against 18million rows?

Curtis

I'm basically wondering what's the best way to query a table with such amount of rows?

I'm guessing I'd need to do some optimization before I even think about queries since anything I try to perform takes around 50 seconds to complete?

Anyone have any ideas on how to speed up my table and perform queries much quicker?

Here's an example query :

SELECT 
    `COMPANY_NAME`, `ZIPCODE`, `ADDRESS`
FROM
    `companies`
WHERE
    replace(`COMPANY_NAME`, ' ', '') = '$area'

Here's my Database structure

id  int(11) PRIMARY
COMPANY_NAME    varchar(255)
ADDRESS varchar(255)
CITY    varchar(255)
STATE   varchar(255)
ZIPCODE varchar(255)
COUNTRY varchar(255)
WEB_ADDRESS varchar(255)
PHONE_NUMBER    varchar(255)
FAX_NUMBER  varchar(255)
CONTACT_NAME    varchar(255)
TITLE   varchar(255)
GENDER  varchar(255)
EMPLOYEE    varchar(255)
SALES   varchar(255)
MAJOR_DIVISION_DESCRIPTION  text
SIC_2_Code_Description  text
SIC_4_Code  varchar(255)
SIC_4_Code_Description  text    latin1_swedish_ci

My database is also InnoDB

Gordon Linoff

The key issue on performance on your query is this statement:

WHERE replace(`COMPANY_NAME`, ' ', '') = '$area'

If, instead, you had:

WHERE COMPANY_NAME = '$area'

then an index on COMPANY_NAME could be used. However, because your version uses a function on the column, the database engine has to do a full table scan instead of an index lookup.

More information would be needed to solve this problem. One would be to update COMPANY_NAME to remove spaces and add an index on it. Another would be to add another column (as suggested by @dystroy). If you are only trying to remove trailing spaces, then you probably want to update the existing column. Finally, if you are trying to match words in COMPANY_NAME, and not the whole name, then a full text index may be the best solution.

By the way, have you tried this?

WHERE COMPANY_NAME like '$area%'

That is, do a wildcard search at the end of $area. That can use an index on COMPANY_NAME and, if it returns the right results, might be the easiest way to solve your problem.

Collected from the Internet

Please contact [email protected] to delete if infringement.

edited at
0

Comments

0 comments
Login to comment

Related

From Dev

How to update all the rows against each id

From Dev

How can I query against a concatenated field?

From Dev

How to query against a large list of words with an IN clause

From Dev

How to query against many tables in mySQL

From Dev

How to do this query against MySQL database table?

From Dev

MATCH many rows against many rows within SQL query and find percentile match?

From Dev

Query optimization on a Table with 1 Million rows

From Dev

Query large table with 50 million rows

From Dev

How to do a EntityFramework query against md5 of a column

From Dev

How do I map an OData query against a DTO to another entity?

From Dev

How do I map an OData query against a DTO to an EF entity?

From Dev

How could I check a single document against an existing mongoose query?

From Dev

AppEngine/NDB How to apply a query filter against single entity

From Dev

how to run sparql query against local rdf file in php

From Dev

How do I run a foreach against an array and include it in an SQL query?

From Dev

How to compare Kusto query against 2 table columns?

From Dev

How to search text using MATCH...AGAINST query?

From Dev

How to query against dbo.AspNetUserLogins table (IdentityUserLogin entity)?

From Dev

How to run the same query against multiple tables in the database

From Dev

how to query against a many to many relation with entity framework 6

From Dev

How to find the reason for the difference in the execution time of a query against different databases?

From Dev

How to hide rows of a query of MySQL?

From Dev

How to hide rows of a query of MySQL?

From Dev

Optimizing SQL query on table of 10 million rows: neverending query

From Dev

How do I compare rows of a table against all other rows of the table?

From Dev

Query against XML columns

From Dev

Optimal MYSQL query for longest prefix matching in a table with 5 million rows

From Dev

Slow Query on Medium MySQL Table (1 Million Rows)

From Dev

Optimal MYSQL query for longest prefix matching in a table with 5 million rows

Related Related

  1. 1

    How to update all the rows against each id

  2. 2

    How can I query against a concatenated field?

  3. 3

    How to query against a large list of words with an IN clause

  4. 4

    How to query against many tables in mySQL

  5. 5

    How to do this query against MySQL database table?

  6. 6

    MATCH many rows against many rows within SQL query and find percentile match?

  7. 7

    Query optimization on a Table with 1 Million rows

  8. 8

    Query large table with 50 million rows

  9. 9

    How to do a EntityFramework query against md5 of a column

  10. 10

    How do I map an OData query against a DTO to another entity?

  11. 11

    How do I map an OData query against a DTO to an EF entity?

  12. 12

    How could I check a single document against an existing mongoose query?

  13. 13

    AppEngine/NDB How to apply a query filter against single entity

  14. 14

    how to run sparql query against local rdf file in php

  15. 15

    How do I run a foreach against an array and include it in an SQL query?

  16. 16

    How to compare Kusto query against 2 table columns?

  17. 17

    How to search text using MATCH...AGAINST query?

  18. 18

    How to query against dbo.AspNetUserLogins table (IdentityUserLogin entity)?

  19. 19

    How to run the same query against multiple tables in the database

  20. 20

    how to query against a many to many relation with entity framework 6

  21. 21

    How to find the reason for the difference in the execution time of a query against different databases?

  22. 22

    How to hide rows of a query of MySQL?

  23. 23

    How to hide rows of a query of MySQL?

  24. 24

    Optimizing SQL query on table of 10 million rows: neverending query

  25. 25

    How do I compare rows of a table against all other rows of the table?

  26. 26

    Query against XML columns

  27. 27

    Optimal MYSQL query for longest prefix matching in a table with 5 million rows

  28. 28

    Slow Query on Medium MySQL Table (1 Million Rows)

  29. 29

    Optimal MYSQL query for longest prefix matching in a table with 5 million rows

HotTag

Archive