Join with more than 15 fields (1 Viewer)

jaryszek

Registered User.
Local time
Today, 07:38
Joined
Aug 25, 2016
Messages
756
Hi,

i have SQL like here:

Code:
SELECT DISTINCT *
FROM tblVolumes AS t2 LEFT JOIN qryIm_SourceVolumes AS t1 ON (t2.PercentSnapshotSpace = t1.PercentSnapshotSpace or (t2.PercentSnapshotSpace is Null AND t1.PercentSnapshotSpace is Null)) AND (t2.FilesystemType = t1.FilesystemType or (t2.FilesystemType is Null AND t1.FilesystemType is Null)) AND (t2.DiskNumber = t1.DiskNumber or (t2.DiskNumber is Null AND t1.DiskNumber is Null)) AND (t2.VolumeNameIDFK = t1.VolumeNameIDFK or (t2.VolumeNameIDFK is Null AND t1.VolumeNameIDFK is Null)) AND (t2.PhysicalDiskModelIDFK = t1.PhysicalDiskModelIDFK or (t2.PhysicalDiskModelIDFK is Null AND t1.PhysicalDiskModelIDFK is Null)) AND (t2.TierID = t1.TierID or (t2.TierID is Null AND t1.TierID is Null)) AND (t2.Ordinal = t1.Ordinal or (t2.Ordinal is Null AND t1.Ordinal is Null)) AND (t2.EnvironmentID = t1.EnvironmentID or (t2.EnvironmentID is Null AND t1.EnvironmentID is Null)) AND (t2.ShareMode = t1.ShareMode or (t2.ShareMode is Null AND t1.ShareMode is Null)) AND (t2.SubjectComponent = t1.SubjectComponent or (t2.SubjectComponent is Null AND t1.SubjectComponent is Null)) AND (t2.FormatCommand = t1.FormatCommand or (t2.FormatCommand is Null AND t1.FormatCommand is Null)) AND (t2.Description = t1.Description or (t2.Description is Null AND t1.Description is Null)) AND (t2.Function = t1.Function or (t2.Function is Null AND t1.Function is Null)) AND (t2.Caching = t1.Caching or (t2.Caching is Null AND t1.Caching is Null)) AND (t2.writeAcceleratorEnabled = t1.writeAcceleratorEnabled or (t2.writeAcceleratorEnabled is Null AND t1.writeAcceleratorEnabled is Null)) AND (t2.Caching = t1.Caching or (t2.Caching is Null AND t1.Caching is Null))
WHERE (((t1.Ordinal) Is Null) AND ((t1.EnvironmentID) Is Null) AND ((t1.TierID) Is Null) AND ((t1.PhysicalDiskModelIDFK) Is Null) AND ((t1.VolumeNameIDFK) Is Null) AND ((t1.DiskNumber) Is Null) AND ((t1.FileSystemType) Is Null) AND ((t1.PercentSnapshotSpace) Is Null) AND ((t1.ShareMode) Is Null) AND ((t1.SubjectComponent) Is Null) AND ((t1.FormatCommand) Is Null) AND ((t1.Description) Is Null) AND ((t1.Function) Is Null) AND ((t1.Caching) Is Null) AND ((t1.writeAcceleratorEnabled) Is Null));

so i have more than 16 fields and this is working. The restriction of joins in Access 16 is not working here.

Why ?

What i am not understanding?

Best,
Jacek
 

Galaxiom

Super Moderator
Staff member
Local time
Tomorrow, 00:38
Joined
Jan 20, 2009
Messages
12,851
You don't have more than 15 joins. Instead you have one incredibly complex join that suggests an inappropriate data structure.
 

jaryszek

Registered User.
Local time
Today, 07:38
Joined
Aug 25, 2016
Messages
756
thank you Galaxiom.

Why data is inappropriate?
I have just very very big csv table when i am importing data. This table has more than 16 fields. Because the characteristic (Volume from Azure) can have more than 16 fields.

What is bad here? I can not enforce data to be normalized if attibutes are describing specific characteristic.

Jacek
 

MajP

You've got your good things, and you've got mine.
Local time
Today, 10:38
Joined
May 21, 2018
Messages
8,516
Code:
SELECT DISTINCT * 
FROM   tblvolumes AS t2 
       LEFT JOIN qryim_sourcevolumes AS t1 
              ON ( t2.percentsnapshotspace = t1.percentsnapshotspace 
                    OR ( t2.percentsnapshotspace IS NULL 
                         AND t1.percentsnapshotspace IS NULL ) ) 
                 AND ( t2.filesystemtype = t1.filesystemtype 
                        OR ( t2.filesystemtype IS NULL 
                             AND t1.filesystemtype IS NULL ) ) 
                 AND ( t2.disknumber = t1.disknumber 
                        OR ( t2.disknumber IS NULL 
                             AND t1.disknumber IS NULL ) ) 
                 AND ( t2.volumenameidfk = t1.volumenameidfk 
                        OR ( t2.volumenameidfk IS NULL 
                             AND t1.volumenameidfk IS NULL ) ) 
                 AND ( t2.physicaldiskmodelidfk = t1.physicaldiskmodelidfk 
                        OR ( t2.physicaldiskmodelidfk IS NULL 
                             AND t1.physicaldiskmodelidfk IS NULL ) ) 
                 AND ( t2.tierid = t1.tierid 
                        OR ( t2.tierid IS NULL 
                             AND t1.tierid IS NULL ) ) 
                 AND ( t2.ordinal = t1.ordinal 
                        OR ( t2.ordinal IS NULL 
                             AND t1.ordinal IS NULL ) ) 
                 AND ( t2.environmentid = t1.environmentid 
                        OR ( t2.environmentid IS NULL 
                             AND t1.environmentid IS NULL ) ) 
                 AND ( t2.sharemode = t1.sharemode 
                        OR ( t2.sharemode IS NULL 
                             AND t1.sharemode IS NULL ) ) 
                 AND ( t2.subjectcomponent = t1.subjectcomponent 
                        OR ( t2.subjectcomponent IS NULL 
                             AND t1.subjectcomponent IS NULL ) ) 
                 AND ( t2.formatcommand = t1.formatcommand 
                        OR ( t2.formatcommand IS NULL 
                             AND t1.formatcommand IS NULL ) ) 
                 AND ( t2.description = t1.description 
                        OR ( t2.description IS NULL 
                             AND t1.description IS NULL ) ) 
                 AND ( t2.function = t1.function 
                        OR ( t2.function IS NULL 
                             AND t1.function IS NULL ) ) 
                 AND ( t2.caching = t1.caching 
                        OR ( t2.caching IS NULL 
                             AND t1.caching IS NULL ) ) 
                 AND ( t2.writeacceleratorenabled = t1.writeacceleratorenabled 
                        OR ( t2.writeacceleratorenabled IS NULL 
                             AND t1.writeacceleratorenabled IS NULL ) ) 
                 AND ( t2.caching = t1.caching 
                        OR ( t2.caching IS NULL 
                             AND t1.caching IS NULL ) ) 
WHERE  ( ( ( t1.ordinal ) IS NULL ) 
         AND ( ( t1.environmentid ) IS NULL ) 
         AND ( ( t1.tierid ) IS NULL ) 
         AND ( ( t1.physicaldiskmodelidfk ) IS NULL ) 
         AND ( ( t1.volumenameidfk ) IS NULL ) 
         AND ( ( t1.disknumber ) IS NULL ) 
         AND ( ( t1.filesystemtype ) IS NULL ) 
         AND ( ( t1.percentsnapshotspace ) IS NULL ) 
         AND ( ( t1.sharemode ) IS NULL ) 
         AND ( ( t1.subjectcomponent ) IS NULL ) 
         AND ( ( t1.formatcommand ) IS NULL ) 
         AND ( ( t1.description ) IS NULL ) 
         AND ( ( t1.function ) IS NULL ) 
         AND ( ( t1.caching ) IS NULL ) 
         AND ( ( t1.writeacceleratorenabled ) IS NULL ) );
 

Galaxiom

Super Moderator
Staff member
Local time
Tomorrow, 00:38
Joined
Jan 20, 2009
Messages
12,851
Why data is inappropriate?

You are battling with some horrible source data that doesn't include a key.

You have done all you can do and I am surprised it doesn't break already. Be thankful rather than question why it works.

Is there something we can help with?
 

jaryszek

Registered User.
Local time
Today, 07:38
Joined
Aug 25, 2016
Messages
756
You are battling with some horrible source data that doesn't include a key.

This is data from csv - it is added manually by user.
This is standard if import data do not have the key.

You can read about approach for importing here:

http://www.mendipdatasystems.co.uk/synchronise-data-2/4594514002

Only difference is that table is very big.

Is there something we can help with?
thank you, nothing more.

Hi MajP,
why did you do this sql in such a nice form?
btw. it is possible to have this nice formatting in Access query directly?

Best,
Jacek
 

jaryszek

Registered User.
Local time
Today, 07:38
Joined
Aug 25, 2016
Messages
756
I have question:
I created query (sorry fields are from two entirely different tables - i wnat to show only mechanism):





and as you can see there are 3 joins on fields in query design but still in SQL statement there is only one left join statement.
So restriction for 15 joins concerns having 15 times in SQL statement LEFT JOIN?

What does the 15 joins restriction is all about? I am confused now.
Can anybody help and explain?

Best,
Jacek
 

Attachments

  • Screenshot_2.png
    Screenshot_2.png
    87.7 KB · Views: 458
  • Screenshot_3.png
    Screenshot_3.png
    73.6 KB · Views: 457
Last edited:

isladogs

MVP / VIP
Local time
Today, 15:38
Joined
Jan 14, 2017
Messages
18,207
No. The SQL statement indicates three fields are left joined as does the query designer.

I know I keep saying this but we discussed the join limit issue at length a few months ago in several threads started by you. If I recall correctly, previously you were trying unsuccessfully to run action queries with 17 joins
Having rechecked Access specifications it says the maximum is 16 joins in a query (possibly less if using MVFs).
See https://support.office.com/en-us/article/Access-specifications-0cf3c66f-9cf2-4e32-9568-98c1025bb47c

I haven't counted your joins but it would appear you are on or close to the limit. As a result, I think you are going to repeatedly get error messages related to the number of joins unless you can find another way of managing this data
 

The_Doc_Man

Immoderate Moderator
Staff member
Local time
Today, 09:38
Joined
Feb 28, 2001
Messages
27,123
jaryszek, the JOIN limit is on how many TABLES you can JOIN in a single query but a different limit applies to how many fields you can have in a query. You can only generate a recordset with 255 fields in it, and what you showed us doesn't seem to go that far. There are limits to how many AND conjunctions you can have, but again, you don't appear to approach that limit.
 

Galaxiom

Super Moderator
Staff member
Local time
Tomorrow, 00:38
Joined
Jan 20, 2009
Messages
12,851
it is possible to have this nice formatting in Access query directly?

You can do it manually in Access but give it half a chance and it will scramble it all again.

When I have a complex query in Access SQL I tack a UNION to a dummy table onto the end. Access designer can't handle unions so it leaves the formatting alone.

Mostly I write queries in MSSQL Server Management Studio now and it is such a pleasure.
 

jaryszek

Registered User.
Local time
Today, 07:38
Joined
Aug 25, 2016
Messages
756
Hi Colin,

i know that we have similar topic but:

The SQL statement indicates three fields are left joined as does the query designer.

hmm sorry, i do not know how but i overcalculated number of joins...

oo you have right, here i have 16 joins! So max! I though that i have more than 16...uff i was so confused.

jaryszek, the JOIN limit is on how many TABLES you can JOIN in a single query but a different limit applies to how many fields you can have in a query. You can only generate a recordset with 255 fields in it, and what you showed us doesn't seem to go that far. There are limits to how many AND conjunctions you can have, but again, you don't appear to approach that limit.

i see conflict here. I thought that limit is about how many fields in one query you can join. Not tables?

Can you be consistent?


thank you!

Jacek
 

isladogs

MVP / VIP
Local time
Today, 15:38
Joined
Jan 14, 2017
Messages
18,207
Jacek
There are several limits you may hit. As soon as you hit any of them the query won't work
The answer will therefore depend ...it possibly will be different using outer joins or filtering to avoid null values or if running action queries or if your table(s) have no primary key e.g. CSV file
As I NEVER need to create queries with this many joins, all I can do is exactly the same as you can do yourself:
a) Look in Access specifications which states a limit of 16 joins (see earlier posts)
b) Trial & error adding more field joins and/or more table joins

Attached are screenshots of two simple test queries created solely for this reply
1. Two 'copies' of the same table as a self join - with 23 fields joined
2. 20 tables joined with single inner join to the main table

Both worked...though unsurprisingly the second wasn't very fast
No error messages in either case
If I had done the second test slighlty differently, perhaps I would have hit the error.

So my tests don't give you the simple answer you want.
But I don't think there is a simple answer.
As I've said already, you need to try and find ways of doing your task slightly differently so the problem no longer arises
 

Attachments

  • qryJoinTest1.PNG
    qryJoinTest1.PNG
    24.9 KB · Views: 251
  • qryJoinTest2.jpg
    qryJoinTest2.jpg
    97.2 KB · Views: 254
Last edited:

CJ_London

Super Moderator
Staff member
Local time
Today, 15:38
Joined
Feb 19, 2013
Messages
16,601
I had an issue a while back to identify duplicates in transaction data - data came from multiple sources and no primary key in any of them - and about 30 fields where the duplicate might occur.

My solution was to 'package' groups of the relevant fields into another query which output a single key value by hashing the field values.

There were two such queries, one to work on the new transaction data and one to work on the destination table - although ultimately I stored the hash values in the destination table, indexed - no duplicates
 

jaryszek

Registered User.
Local time
Today, 07:38
Joined
Aug 25, 2016
Messages
756
thank you Colin for tests and such a good explanation. Wow!

Awesome man!

CJ_London,
thank you, i loved MajP solution to find matches but your solutions seems to be good one.

Best,
Jacek
 

The_Doc_Man

Immoderate Moderator
Staff member
Local time
Today, 09:38
Joined
Feb 28, 2001
Messages
27,123
i see conflict here. I thought that limit is about how many fields in one query you can join. Not tables?

Can you be consistent?

The way I read the "specifications and limits" articles, and based on my understanding of the nomenclature, a JOIN between two tables with more than one field to test between each table is merely one JOIN between two tables with (in your case) three conditions to match between the members of the JOIN.

Of greater concern is that when doing JOINs you would do best to have indexes on any field that would be participating in the JOIN, but you have limits on how many fields in a single table can have an index. Of even greater importance, there is also a limit-per-query of how many relationships can be enforced where a relationship exists. You WANT to have relationships because the query builder is smart enough to see them and use them, but there IS such a thing as "too many relationships."
 

jaryszek

Registered User.
Local time
Today, 07:38
Joined
Aug 25, 2016
Messages
756
thank you The_Doc_Man very much!

Colin checked 2 cases and still this is working so hard to say and have one answer for this.

What i can do is to change
Code:
( t2.percentsnapshotspace = t1.percentsnapshotspace 
                    OR ( t2.percentsnapshotspace IS NULL 
                         AND t1.percentsnapshotspace IS NULL ) )

to

Code:
 Nz(t2.percentsnapshotspace,"NULL") = Nz(t1.percentsnapshotspace,"NULL")

thank you very much for help and support,
Jacek
 

Users who are viewing this thread

Top Bottom