Solved Auto Index (1 Viewer)

Space Cowboy

Member
Local time
Today, 10:34
Joined
May 19, 2024
Messages
193
Good Afternoon Good People

I use a make table Query to make a table which I then reference in another query.
To reference the table in the subsequent query I need to manually add an auto index field.
Is there anything I can add to SQL in the make table query which will add and auto index field to the table?
 

GPGeorge

George Hepworth
Local time
Today, 02:34
Joined
Nov 25, 2004
Messages
2,168
Please explain what you mean by adding an auto index field to a table.

Start with your definition of "auto index field", please.
 

plog

Banishment Pending
Local time
Today, 04:34
Joined
May 11, 2011
Messages
11,730
I use a make table Query to make a table which I then reference in another query.

Why? Why must that query's data exist in a table? Why can't you just make it a SELECT query and use that in the next query?
 

Space Cowboy

Member
Local time
Today, 10:34
Joined
May 19, 2024
Messages
193
I think That I have probably used incorrect terminology.

CaptureDB1.PNG

I need to create the field LSR_ID into the table.
 

Space Cowboy

Member
Local time
Today, 10:34
Joined
May 19, 2024
Messages
193
Why? Why must that query's data exist in a table? Why can't you just make it a SELECT query and use that in the next query?

if I run the second query from the first it is very slow, 1.7 million lines with 2 Dcounts.
If I make the table first the second query is much faster as it doesn't seem to have to recalculate first query. I used the table, rightly or wrongly, to "fix the data" so it runs faster in second query
 

jdraw

Super Moderator
Staff member
Local time
Today, 05:34
Joined
Jan 23, 2006
Messages
15,440
SpaceCowboy,
Can you define the table with the fields and PK you need and then append records to that table separately?
 

Space Cowboy

Member
Local time
Today, 10:34
Joined
May 19, 2024
Messages
193
SpaceCowboy,
Can you define the table with the fields and PK you need and then append records to that table separately?
I am sorry Jdraw but I have not a clue what the process is that you are trying to explain to me.

The make table query puts the fields into the table. I have to manually add the auto number field as the next query will not run without that field.
I just thought, rather in hope than expectation, that I could add a line or something into the SQL of my maketable query.
 

jdraw

Super Moderator
Staff member
Local time
Today, 05:34
Joined
Jan 23, 2006
Messages
15,440
My thought was this:
-manually create the table you need
-supply each field and type
-identify the proper field as the PK
-now the table exists
-next run your query to append records to the new table.

OR programmatically vba code along this set up to create your table and autonumber Primary Key
Code:
Dim Createsql As String
Createsql = "Create Table tbl_Tracks" _
          & "(TTId autoincrement primary key,Tracklink  Number, ArtistLink Number, IsAvailable Bit);"
          CurrentDb.Execute Createsql, dbFailOnError
 

Space Cowboy

Member
Local time
Today, 10:34
Joined
May 19, 2024
Messages
193
@jdraw

I can follow the logic there apart from "TTid"

Would it be possible to create the Table and run the SQL to populate it?

It means learning something completely new and it pleases me that you think I am capable of that, but I have only just got it working after several weeks of frustration through SQL.

I was thinking of running a Macro to run several of the make table queries automatically and a deviation onto VBA would be outside my current level of understanding.
 

CJ_London

Super Moderator
Staff member
Local time
Today, 10:34
Joined
Feb 19, 2013
Messages
16,799
VBA would be outside my current level of understanding.
So use your make table to create the table, then go into design view and add an autonumber field

whilst you are at it, also add indexes to the fields your subsequent queries will be using as criteria, sorting or joining on
 

GPGeorge

George Hepworth
Local time
Today, 02:34
Joined
Nov 25, 2004
Messages
2,168
@jdraw

I can follow the logic there apart from "TTid"

Would it be possible to create the Table and run the SQL to populate it?

It means learning something completely new and it pleases me that you think I am capable of that, but I have only just got it working after several weeks of frustration through SQL.

I was thinking of running a Macro to run several of the make table queries automatically and a deviation onto VBA would be outside my current level of understanding.
First, thank you for providing the context.

You will eventually need to use VBA for any serious Access development; macros are okay for the light end stuff but they are inherently limited. Here, you can accomplish what you need to do more readily with VBA, so now is a good time to expand your knowledge.

You can create a table once. Then reuse it, as previously suggested. I liken it to building a wooden box to hold firewood for your fireplace. If you create a new box each time you want to bring in firewood, you have to invest extra time and effort to do that. The alternative is to reuse the old one and just refill it as needed.
 

ebs17

Well-known member
Local time
Today, 11:34
Joined
Feb 7, 2020
Messages
2,090
Code:
sSQL = "SELECT * INTO NewTable FROM OldTable"
db.Execute sSQL, dbFailOnError
sSQL = "ALTER TABLE NewTable ADD COLUMN Idx COUNTER"
db.Execute sSQL, dbFailOnError
sSQL = "CREATE INDEX idx_name ON NewTable (Idx)"
db.Execute sSQL, dbFailOnError
So you can create the field later and also index it, all within a VBA procedure.

However, since redundant data and saved calculation results are created here, the entire process should be classified in the temporary tables area. The interim use of temporary tables can very quickly lead to a simplification of query runs, and targeted indexing/re-indexing can contribute greatly to performance.
 

The_Doc_Man

Immoderate Moderator
Staff member
Local time
Today, 04:34
Joined
Feb 28, 2001
Messages
27,729
If you create the table first, empty but with all fields including your auto-number defined, then use an Append type query to populate your table second, the auto-number will populate correctly in that process. But by making the table first without the field and then going back to add the auto-number field, you force Access to go back and revisit EVERY RECORD - because by adding the new field as a second step, you force it to rewrite every already-existing record to have a populated auto-number. Access tends to "squeeze" records together tightly with no slack space between records. Adding that auto-number, every record just got one field larger so has to be copied and repopulated. Plus, Access has to go back and delete the now-obsolete original record. That combination has to be slower than molasses in the Arctic Circle.

But I'm more interested in the reason for this. The "first query" extracts data from somewhere. Is that a table? Because it seems to me that, rather than building a "helper table", you should be running the "second query" to directly compute your counts from the original source, which would be the right answer. But you can't really do that because of structural issues in the original database that is "too big to fail." I might point out that if it is this hard to gather what you wanted, it may already be failing. Failing to support the company's needs.

I recall that you are working with a non-Access back-end from a different machine. You are grabbing data from an ODBC source, Visual Foxpro. By making a local table first, you make the "data fetch" portion of subsequent operations local rather than networked. That can certainly make a profound difference in speed so, for efficiency, this might still be your best bet.

So you have created this "helper table." You run that next query that requires a suitable auto-number field. What do you do with the "helper table" when you are done with it? Particularly if it involved 1.7 million records, that is a LOT of database bloat if you are just going to delete it. Looking back through your prior threads, this is something you want to run every day. You are going to have such a bad bloat problem very quickly. Do you run frequent Compact & Repair operations? Or do you start fresh each day by copying your DB to a work area and only using the "working copy," not the "master copy"?

This thread also seems to be a near-direct continuation of another thread:


When the question changes, there are reasons for starting a new thread and that is perfectly OK. But when the question hasn't changed, you COULD continue in the same thread. I know you are struggling here with bulky data and the need to extract statistics from a complicated source. You are also fighting an environment that (based on your comments) is new to you.

I'm not trying to give you a hard time. I'm just commenting that if the topic hasn't changed, splitting the question now makes us look in more than one place to gain any sense of continuity. It's a fine line to cross and you are DEFINITELY not the first person to have faced the problem of when to start a new thread. Call this a friendly side comment that sometimes continuity is more important and sometimes a change IS needed. The trick is to recognize that "New/Continue line" when you approach it.
 

Pat Hartman

Super Moderator
Staff member
Local time
Today, 05:34
Joined
Feb 19, 2002
Messages
44,043
if I run the second query from the first it is very slow, 1.7 million lines with 2 Dcounts.
That makes 3.4 million queries created on the fly. Sure is a lot of overhead. Use a totals query instead. Lots less overhead and only a single query runs instead of millions of separate queries.
 

Space Cowboy

Member
Local time
Today, 10:34
Joined
May 19, 2024
Messages
193
hello Pat

I was previously advised that DCOUNT was the way to go.
This is my SQL

SELECT ftq21.odate,
filtertable2.orderid,
ftq21.sumofunitprice,
Dcount("*", "ftq21", "sumofunitprice >=" & ftq21.sumofunitprice &
" and odate = " & Format(
filtertable2.odate, "\#yyyy-mm-dd\#"))
AS dayrank,
filtertable2.unitprice,
Dcount("*", "filtertable2", "unitprice >=" & filtertable2.unitprice &
" and orderid= " &
filtertable2.orderid &
" and odate = " &
Format(filtertable2.odate, "\#yyyy-mm-dd\#")) AS
PriceRank
FROM filtertable2
INNER JOIN ftq21
ON filtertable2.orderid = ftq21.orderid
ORDER BY ftq21.odate,
ftq21.sumofunitprice DESC,
filtertable2.unitprice DESC;

Will Totals work on this?
 

Space Cowboy

Member
Local time
Today, 10:34
Joined
May 19, 2024
Messages
193
Hello Cheekybuddha.

SQL for FTQ21 is as follows

SELECT FilterTable2.odate, FilterTable2.OrderId, Sum(FilterTable2.UnitPrice) AS SumOfUnitPrice
FROM FilterTable2
GROUP BY FilterTable2.odate, FilterTable2.OrderId;

Thanks for your input.
 

The_Doc_Man

Immoderate Moderator
Staff member
Local time
Today, 04:34
Joined
Feb 28, 2001
Messages
27,729
I wish to clarify something I implied in post #13.

When you are creating an auto-number field and adding it to an existing table with the intent that it be populated, design the table empty and then populate it. Auto-numbers are a special case AND the indexes are monotonically increasing.

HOWEVER, if the field to be indexed is NOT auto-numbered, your better bet is to build the table non-indexed and then come back to add an index (for an already populated field.) The difference is that in the non-auto-number field, the indexed field's values are not predictable. In that case, it is easier to build the index after-the-fact because less memory is being "churned" in the process.

I had to look that one up before I offered the correction.
 

CJ_London

Super Moderator
Staff member
Local time
Today, 10:34
Joined
Feb 19, 2013
Messages
16,799
I presume oDate and orderID are indexed? They should be

you should be able to substitute your dcounts with a subquery, For example will get rid of a format and will probable be faster

Dcount("*", "ftq21", "sumofunitprice >=" & ftq21.sumofunitprice &
" and odate = " & Format(
filtertable2.odate, "\#yyyy-mm-dd\#"))
AS dayrank,

becomes

SQL:
(SELECT COUNT *
FROM ftq21 as D
WHERE D.sumofunitprice >=ftq21.sumofunitprice  and D.odate = filtertable2.odate)
AS dayrank,


and
Dcount("*", "filtertable2", "unitprice >=" & filtertable2.unitprice &
" and orderid= " &
filtertable2.orderid &
" and odate = " &
Format(filtertable2.odate, "\#yyyy-mm-dd\#")) AS
PriceRank

becomes

SQL:
(SELECT Count * 
FROM filtertable2 as F
WHERE F.unitprice >=filtertable2.unitprice and F.orderid=filtertable2.odate)
AS PriceRank
 

Users who are viewing this thread

Top Bottom