help needed tricky issue

[Minty]

Make sure any anti-virus isn't scanning your database folder.

Check windows firewall settings.

 

[dtdukes86]

so after using a msg box popup between each line of code ;-)

i no longer think this is a networking speed issue but rather the way the individual computer handles the code. the code on each system is the same so it must be something to do with the software installation somewhere maybe something daft like an update who knows? in either case, it appears the issue is with a local function rather than writing the data back to the shared backend

 

[GPGeorge]

so after using a msg box popup between each line of code ;-)

i no longer think this is a networking speed issue but rather the way the individual computer handles the code. the code on each system is the same so it must be something to do with the software installation somewhere maybe something daft like an update who knows? in either case, it appears the issue is with a local function rather than writing the data back to the shared backend

That lends more credence to the suggestions @Minty offered regarding AV, or other software on that computer.

 

[Isaac]

Well, I was aiming at a different aspect.

I was not precise enough; the VM itself had around 25 GB available in total for the mdf and ldf files.

Once we hit that, it stopped cold. The problem was it was a development database so there was a lot of churn going on, growing the ldf beyond what IT ever thought we'd need. Our process involved downloading huge files from an Oracle OLT database for analysis. One or two of those ETL processes filled up the log file rapidly.

I worked once for a regional department of revenue (taxing) and we got data shared from the Feds with us - basically all Fed data on w2's, 1099's, etc. The files were billions of records-text files. Talk about needing special editions of ultraedit and notepad++ and the dba's unable to believe how much space we needed!
Absolutely phenomenal, though, how fast SSIS loaded that data. Watching that green thing spin and going into the hundreds of millions within a few minutes is amazing

 

[dtdukes86]

quick question is there any major difference in speed between sql exp and sql developer and which do u guys recocomend.

 

[Minty]

Developer edition is a fully functional version of SQL Server Enterprise, but only allows development use, not production use.
SQL Server Express is not as fully featured and has constraints on the amount of memory it will use, and the size of the database compared to the paid for editions.
There is a proper comparison table here, with complete descriptions.

Editions and supported features of SQL Server 2022 - SQL Server

Learn details of the features supported by the various editions of SQL Server 2022.

learn.microsoft.com

 

[isladogs]

@dtdukes86
I've just discovered the email you sent me a few days ago and sent a reply.
Unfortunately, I'm not sure I can offer much that hasn't already been covered here

In my reply to your email, I asked whether the slow machines are running 32-bit Access and, if so, suggested adding large address awareness.
If that's not relevant, then consider a different file format for the address downloads from Ideal Postcodes e.g. CSV or XML instead of JSON.

However, like the others who have responded, I'm almost certain that the issue lies somewhere on your network

You talked about the speed of large data transfers on your network but the JSON address files involved with this app are tiny Typically 10-20 kB only.
I've attached a zipped JSON address file for a randomly selected postcode as an example

Downloading and processing files such as this really shouldn't cause any issue with modern hardware.

 

[arnelgp]

i am sure you can get a full refund!

 

[dtdukes86]

i am sure you can get a full refund!

hahaha i doubt that my employer is footing the bill dont worry ;-) and ive literally pulled the software apart so.... ;-)