Optimum method of handling subforms from large SQL data sets (1 Viewer)

Zedster

Registered User.
Local time
Today, 12:24
Joined
Jul 2, 2019
Messages
169
I have a SQL BE/Access FE database for relating POs and Invoices to Budget Lines.

A master form for the budget lines uses a pass through SQL query. It has two view only subforms (each in a tab) that filter to show related POs and Invoices. The PO table has approx 7000 records, the invoices 12000 records.

When I open the form, it takes an age to display.

I would love to have the suforms driven by pass through SQL queries but Access doesn't allow that :banghead: It appears that the best way to speed it up is to have the form's "on open" event create local tables from parameter queries, then use these for the subforms.

The question is what is the best way from performance perspective.

1. Each open delete previous table and make new table
2. Leave table and each open delete contents and append new contents

Alternatively does anyone have a better suggestion to avoid pulling 19000 records across the server.
 

CJ_London

Super Moderator
Staff member
Local time
Today, 12:24
Joined
Feb 19, 2013
Messages
16,606
Alternatively does anyone have a better suggestion to avoid pulling 19000 records across the server.
depends on what you are doing with the data once you have it. Your performance issue is certainly to do with the volume of records.

I would love to have the suforms driven by pass through SQL queries but Access doesn't allow that
not sure what you mean by this - you can have a passthrough query as a recordsource to a form. Another option is to use stored procedures but not sure that would benefit you in this instance - as above, depends what you are doing with the data
 

Minty

AWF VIP
Local time
Today, 12:24
Joined
Jul 26, 2013
Messages
10,366
Have you tried loading the master form blank ? Then you won't get any data on the sub forms, and it should load quickly.

Alternatively, Only load the sub forms data sources after the main form has a record?

As CJ said - a lot depends on what you are doing with the data.
 

Zedster

Registered User.
Local time
Today, 12:24
Joined
Jul 2, 2019
Messages
169
not sure what you mean by this - you can have a passthrough query as a recordsource to a form.

All my forms that don't need to be edited use pass through queries for their recordsource to speed up performance. But when I set a passthrough query as the recordsource for a subform I get an error "
You can't use a pass-through query as a recordsource for a subform". According to google I am not alone with this problem there are many forum posts from users that have the same experience and the concensus appears to be that pass through queries cannot be used for the recordsource of a subform.
If I am doing something wrong or if anyone knows a way to use a passthrough query as a recordsource for a subform that is a perfect solution because that is what I would like to do.
 

Zedster

Registered User.
Local time
Today, 12:24
Joined
Jul 2, 2019
Messages
169
Have you tried loading the master form blank ? Then you won't get any data on the sub forms, and it should load quickly.

Alternatively, Only load the sub forms data sources after the main form has a record?

As CJ said - a lot depends on what you are doing with the data.

OK for what I am trying to do:

I have a table for budgets, each budget has many lines each with a code, purpose and amount (tblBudgetLines). Related many to many to budgets are purchase orders (tblPOs) and related many to one to POs are Invoices (tblInvoices).

For the user, the budgets are displayed in a single form with tabs at the bottom. One tab contains a subform for related POs and another tab contains a subform for related Invoices (which are related to the Budget Line via an inner join query with the parent PO). the subforms are both read only.

A typical budget line may have 5 POs (out of 7000 total) and each of these may have 3 invoices (out of 12000 total).

The recordset for the main form comes from a pass through query to the SqlServer. I would love to use a pass through query for the subforms but I get an error that access does not support pass through queries for sub forms. so the sub forms are created with conventional queries. but this means to display the 5 POs and 3 Invoices in the subform, I am pulling 19000 records across the server. Each time I open the form is takes about 20 seconds to display. If I remove the subforms it is instant.

Searching the internet, this is a problem others have had and the general recommended solution is:

When form is opened use the on open event to run a pass through query to pull the 5 + 3 records. Using VBA make a two new tables and append these 5 _ 3 records and set the recordsource of the subforms to the new tables.

I don't like it, its clunky but I can't find a better solution.

Hence my question was from a performance perspective am I better deleting records from the table and appending new records or deleting the table and making a new table each time from the results of the pass through query.

I am open to all other options. Regarding only displaying sub forms after the main form has opened, I don't see it helping much. the form is only of use when the sub forms are displayed, so I would expect the time to be about the same.
 

Zedster

Registered User.
Local time
Today, 12:24
Joined
Jul 2, 2019
Messages
169
Set Me.Recordset = QueryDefs!mypassthroughqueryname.Recordset

I will give this a try, but I think I tried it a while ago without success (this has been a long standing problem that has got worse recently as the number of POs and Invoices on the server has doubled).
 

Minty

AWF VIP
Local time
Today, 12:24
Joined
Jul 26, 2013
Messages
10,366
20 seconds sounds very slow.

Using the same set up (SQL Backend) I have lots of main forms with subforms that have record counts into the 100,000 's and they load in 1-2 seconds.

In fact one form has about 15 subforms (not all loaded - they are on tabs and mostly loaded on demand), The comments sub form (100000 + records) a customer delivery address subform (20000 + records are loaded immediately. It loads with no records displayed, once a main record has been searched for the form populates in less than 2 seconds.

Have you tried simply linking the tables, and joining the forms with the standard child / master properties?
 

CJ_London

Super Moderator
Staff member
Local time
Today, 12:24
Joined
Feb 19, 2013
Messages
16,606
Others have had and the general recommended solution is:

When form is opened use the on open event
No, the LOAD event

and your passthrough queries would be modified to filter for the 5 or so orders and the 15 or so invoices. Or better still, use a stored procedure

to answer this

Hence my question was from a performance perspective am I better deleting records from the table and appending new records or deleting the table and making a new table each time from the results of the pass through query.
better to delete records from the table, otherwise you will suffer from bloat. From a performance perspective makes little difference.

good luck with your project
 

Pat Hartman

Super Moderator
Staff member
Local time
Today, 07:24
Joined
Feb 19, 2002
Messages
43,213
When you bind the main form to a table or to a query with no selection criteria, you will be bringing every row of every table in all of the subfrms down to the Access client. Essentially, you are doing noting to shift processing to the server. People are always amazed when they convert an Access app with a Jet/ACE BE to SQL server and all of a sudden an app that worked quite well with a Jet/ACE BE slows to a crawl. How can that be?

The solution is to use a query with selection criteria for your main form. Add a couple of text boxes with date fields or combos to give the user a starting point. The bound query will use the criteria fields on the main form header. You can still use a pass-through query but there is no need to. Just use linked tables and you'll be fine. You can also include a join to the mainform record and include criteria for the subforms as well.
 

Zedster

Registered User.
Local time
Today, 12:24
Joined
Jul 2, 2019
Messages
169
The solution is to use a query with selection criteria for your main form. Add a couple of text boxes with date fields or combos to give the user a starting point. The bound query will use the criteria fields on the main form header. You can still use a pass-through query but there is no need to. Just use linked tables and you'll be fine. You can also include a join to the mainform record and include criteria for the subforms as well.

We may be talking at cross purposes because that is exactly what I am already doing. The mainform is filtered to a single record and the recordsource is a pass through query from the SQL Server which pulls a single record. The mainform itself shows the budget line detail it can only be accessed by double clicking a budget line ID on a separate From that lists all budget codes.

I have tried multiple methods of filtering the mainform to a single record including:

Filtering when the calling form opens it from the double click event:

strWhereClause = "BL_ID = '" & Me.BL_ID & "' AND BL_Version = '" & Me.BL_Version & "'"

DoCmd.OpenForm "frmBudgetLineDetailed", acNormal, , strWhereClause, acFormReadOnly, acWindowNormal

Setting using Openargs on the calling form opens it from the double click eventto set up an ACE query for the recordsource as the main form opens:

DoCmd.OpenForm "frmBudgetLineDetailed", acNormal, , , acFormReadOnly, acWindowNormal, Me.BL_ID & "," & BL_Version

Using the double click event of the calling form to create a Passthrough Query QDF using DAO that returns a single record and using this as the recordsource for the main form.

All the above methods work none of them have any bearing on the fact the form takes around 10 seconds to open and Access reports (Not Responding)

This issue is down to the subforms, although these are filtered via the parent to child link of BL_ID, the server pulls 19,000 records and filters on the local machine. I can prove this by deleting the subforms and the main form then opens instantly.

As an experiment I decided to set the recordsource for both these sub forms as a Local passthrough query which is executed on the server. Then I opened each subform as a standalone form and they open fine. but the second I try to open up the main form containing these subforms I get the error shown below:

"You can't use a pass-through query or a non-fixed-column crosstab query as a record source for a subform or subreport"

You can still use a pass-through query ...

The above error occurs if I do. Although interestingly it only occurs for the invoices sub form, the error does not occur for the purchase orders sub form even though both are using pass-through queries.

The pass-through query for the invoices subform is:

SELECT tblInvoicesCurrent.[Invoice Number]
, tblInvoicesCurrent.[Invoice Date]
, tblInvoicesCurrent.Currency
, tblInvoicesCurrent.[Invoice Nett Total]
, [Invoice Nett Total]/[Exchange Rate] AS GBPTotal
, tblInvoicesCurrent.[Invoice Tax]
, tblInvoicesCurrent.[Invoice Total]
, tblInvoicesCurrent.Terms
, tblInvoicesCurrent.Supplier
, tblInvoicesCurrent.[Supplier Code]
, tblInvoicesCurrent.[Receipt Date]
, tblInvoicesCurrent.[Due Date]
, tblInvoicesCurrent.[Posting Date]
, tblInvoicesCurrent.[PO Number]
, tblInvoicesCurrent.[Authorisation Status]
, tblInvoicesCurrent.[Posting Status]
, tblInvoicesCurrent.[Authorised by]
, tblInvoicesCurrent.[Rejected by]
, tblInvoicesCurrent.Department
, tblInvoicesCurrent.Company
, tblInvoicesCurrent.LastUpdated
, tblInvoicesCurrent.UniqueID
, IIf(([BudgetRef] is null),[tblInvoicesCurrent].[BudgetLine],[BudgetRef]) AS Budget
FROM (tblInvoicesCurrent LEFT JOIN tblInvoicesCustomFields ON tblInvoicesCurrent.UniqueID = tblInvoicesCustomFields.InvoiceUniqueID) INNER JOIN tblPurchaseOrdersCurrent ON tblInvoicesCurrent.[PO Number] = tblPurchaseOrdersCurrent.[PO Number]
WHERE IIf(([BudgetRef] is null),[tblInvoicesCurrent].[BudgetLine],[BudgetRef]) = 'ABC123'
 

CJ_London

Super Moderator
Staff member
Local time
Today, 12:24
Joined
Feb 19, 2013
Messages
16,606
a couple of things

1. Currency is a reserved word and should not be used for field names
2. your passthrough query is using VBA functions (iif) which will slow things down. Recommend change it to the server functionality. e.g.

IIf(([BudgetRef] is null),[tblInvoicesCurrent].[BudgetLine],[BudgetRef]) AS Budget

becomes

isnull(BudgetRef,[tblInvoicesCurrent].[BudgetLine]) AS Budget
 

Zedster

Registered User.
Local time
Today, 12:24
Joined
Jul 2, 2019
Messages
169
1. Currency is a reserved word and should not be used for field names

Nice one, I hadn't realised that, all the field names hail from an Excel file which is imported into a SQL table. I replicated the field names from this (hence all the spaces which I personally never use). I will need to think of a work around


2. your passthrough query is using VBA functions (iif) which will slow things down. Recommend change it to the server functionality. e.g.

I googled "Transact SQL Equivalent to IIF" and got the impression it was supported in Transact SQL.


Recommend change it to the server functionality. e.g.

IIf(([BudgetRef] is null),[tblInvoicesCurrent].[BudgetLine],[BudgetRef]) AS Budget

becomes

isnull(BudgetRef,[tblInvoicesCurrent].[BudgetLine]) AS Budget

Time for me to learn. I need to display BudgetRef if it exists, but if it doesn't BudgetLine from the other table. Will your line do that? I read it that it will display BudgetLine if BudgetRef is null else it won't display anything.
 

arnelgp

..forever waiting... waiting for jellybean!
Local time
Today, 19:24
Joined
May 7, 2009
Messages
19,231
IsNull() function only accepts 1 parameter.

IIF(IsNull(BudgetRef), [Forms]![frmAuditTrail]![Area]) As Budget

or use NZ() function:

NZ(BudgetRef, [Forms]![frmAuditTrail]![Area]) As Budget
 

CJ_London

Super Moderator
Staff member
Local time
Today, 12:24
Joined
Feb 19, 2013
Messages
16,606
IsNull() function only accepts 1 parameter.

isnull in TSql should not be confused with the VBA isnull function - TSql isnull is the equivalent of the nz function (another VBA function). See this link https://docs.microsoft.com/en-us/sql/t-sql/functions/isnull-transact-sql?view=sql-server-2017

Any passthrough query using VBA functions will be slow because the (ODBC) driver needs to return the data to be processed in VBA. Care should be taken to ensure that this is kept to a minimum.

I googled "Transact SQL Equivalent to IIF" and got the impression it was supported in Transact SQL.
if you read this link

https://docs.microsoft.com/en-us/sq...nctions-iif-transact-sql?view=sql-server-2017
you will see it translates it to a case statement - which takes time, maybe not a lot, but it still takes time

suggest do some more work on using tsql - these links will indicate some of the pitfalls to avoid
https://www.experts-exchange.com/ar...ccess-Queries-to-SQL-Server-Transact-SQL.html
https://www.databasejournal.com/fea...alents-for-Microsoft-Access-VBA-Functions.htm
 

Pat Hartman

Super Moderator
Staff member
Local time
Today, 07:24
Joined
Feb 19, 2002
Messages
43,213
I wasn't saying to use passthrough queries for the subforms. I was saying to use CRITERIA in the subform's RecordSource query.
 

Zedster

Registered User.
Local time
Today, 12:24
Joined
Jul 2, 2019
Messages
169
After a lot of experimenting (thanks to everyone who contributed with ideas) The problem in the end was down to the IIF, which with 12,000 records was taking an age. Thanks to CJ_London for pointing me in this direction.

2. your passthrough query is using VBA functions (iif) which will slow things down.

I ended up solving it by changing the rationale and data structure of the database so that I didn't need to use IIF to compare the fields in two tables. Speed came down from over 10 seconds to load to less than 1 second. :). Still using pass-through query for main form and conventional client side Access queries for sub-forms.
 

Galaxiom

Super Moderator
Staff member
Local time
Today, 21:24
Joined
Jan 20, 2009
Messages
12,851
I found the best performance on subforms connected to SQL Server came with returning a recordset from a parametrised ADODB Command calling a Stored Procedure.

The parameters are added to the command in the Current Event of the main form and the subform's recordset set to the returned recordset.

I was only displaying records though.
 

Users who are viewing this thread

Top Bottom