Help to import text file of multiple formats (1 Viewer)

lookforsmt

Registered User.
Local time
Today, 14:14
Joined
Dec 26, 2011
Messages
672
HI!

i have similar difficulty to import text file to access. The look of the report is different from the last time i had imported a text file.

i am wondering if there is a way to capture the text details and provide a vba code which can be used for any type of text file imports.

Cause, i have more than 10 text files which are almost of the same look but with different header names.

i am attaching snap shot of the one text file which has many blank columns and only 7 columns with data. There are 3 snap shots which needs to be joint to each other to see from left to right.

any suggestions
 

Attachments

  • Image1_Left.jpg
    Image1_Left.jpg
    102 KB · Views: 139
  • Image2_Mid.jpg
    Image2_Mid.jpg
    104 KB · Views: 134
  • Image3_Right.jpg
    Image3_Right.jpg
    95.8 KB · Views: 122

arnelgp

..forever waiting... waiting for jellybean!
Local time
Today, 18:14
Joined
May 7, 2009
Messages
19,234
maybe the txt file comes from a wide-format report and exported as txt file.
 

Gasman

Enthusiastic Amateur
Local time
Today, 11:14
Joined
Sep 21, 2011
Messages
14,268
I would not be importing report output, but the data that created that report.?
 

jdraw

Super Moderator
Staff member
Local time
Today, 06:14
Joined
Jan 23, 2006
Messages
15,378
It seems like you are importing a report which is processed/formatted for presentation data. If you are trying to make sense of the actual data values, independent of the formatting/possible summarization/grouping etc, then , as gasman suggested, determine if access to the original data is available.
 

lookforsmt

Registered User.
Local time
Today, 14:14
Joined
Dec 26, 2011
Messages
672
Hi! arnlegp, gasman, jdraw, how are you guys, thank you for responding.

arnlegp, you are right, there are two text files which i have copied from two different excels which has columns from A to AQ and A to AS respectively.
(attached 2 txt files)

Gasman & jdraw, i am looking to import from text rather than the excel as it goes into multiple columns and difficult to explain.

i am attaching similar thread which was helped by Majp. The format of the text is different here and i ma finding it difficult to code.

https://www.access-programmers.co.uk/forums/showthread.php?t=300423
 

Attachments

  • OW Sample.txt
    2.5 KB · Views: 113
  • IW Sample.txt
    2.9 KB · Views: 121

CJ_London

Super Moderator
Staff member
Local time
Today, 11:14
Joined
Feb 19, 2013
Messages
16,607
think you will need to read the file line by line and interpret line as required. I have something similar for reading textfiles which have been created using OCR from a scan. you will need to work out how to identify what type of line you are on and perhaps a linecount because you know after 'this' line the next lines are the ones you want.

code would be something like

Code:
Function txtToFile(fPath as string, fName as String)
Dim FileNum As Integer
Dim txtLine As String
Dim lineID As Integer
Dim DataStart As Boolean
Dim dataArr() as string
Dim i as integer
Dim sqlStr
    
    FileNum = FreeFile()
    Open fPath & "\" & fName For Input As #FileNum
    lineID = 0
    DataStart=false
    While Not EOF(FileNum)

        lineID=lineID+1 'just used here to identity lines on the debug
        Line Input #FileNum, txtLine

        if txtLine like "USER ID*" Then 'next lines contain the data I want until line =""
            datastart=true
            Line Input #FileNum, txtLine
            lineID=lineID+1
        elseif txtline="" then 'data finished
            datastart=false
        end if

        if datastart then  'tab char assumed, might be something else

           while instr(txtline,vbTab & vbTab)>0 'assuming there are blank columns separated by multiple tabs
               txtline=replace(txtline,vbTab & vbTab,vbTab) 
           wend

           'assign to the array
           dataarr=split(txtline,vbTab) 

           'lets see what we've got - comment out once working as required
           txtline=""
           for i=0 to ubound(dataarr)
               txtline=txtline & dataarr(i) & "|"
           next i
           debug.print lineID & " : " & txtLine
           'may have issues where values appear in wrong columns, if so then will need additional code to ensure data is in the right columns

           'now do something with it - e.g.
           'sqlstr="INSERT INTO myTable (LoanAcct, DrAcct,....) Values('" & dataarr(0) & "', '" & dataarr(1)...."
           'currentdb.execute sqlstr
        end if

    Wend
    Close #Filenum

End Function
 

jdraw

Super Moderator
Staff member
Local time
Today, 06:14
Joined
Jan 23, 2006
Messages
15,378
I agree with those who have suggested a "line by line parser" suited to your needs. But, I still feel you are dealing with formatted data -much like web screen scraping. If you have access to the underlying data you could put/import that into Access tables. However, if the only option is to post process this formatted data, then the "custom text parser" approach seems to be the favored method.
 
Last edited:

MajP

You've got your good things, and you've got mine.
Local time
Today, 06:14
Joined
May 21, 2018
Messages
8,527
i am wondering if there is a way to capture the text details and provide a vba code which can be used for any type of text file imports.
Cause, i have more than 10 text files which are almost of the same look but with different header names.

I do not think the approach is any different than your original thread. Like CJ said you are always going to read line by line and based on a set of rules determine what to do. So you already have a module that does one formatted report, you just need another set of procedures with different rules for the different reports. The code was written in a vary modular reusable way so there may not be that much new code. As stated in the previous thread it really depends on how consistent the report format is for how well you will be able to parse it. If you remember last time it will take a few iterations of testing to determine all of the possible nuances and account for them. If all 10 reports are pretty similar with slight differences based on the headers, the code may not be too different for each case. So you would either manually tell it to import x type of report or you may be able to code it to first determine what type of report and the parse it correctly. The exercise will have to be the same as last time. Provide the report, determine what you want to parse, determine the rules, test it, and update for real world nuances.

This may be fine in the short run; however, if you have access to the data even if complicated you will likely be able to get a better solution.
 

MajP

You've got your good things, and you've got mine.
Local time
Today, 06:14
Joined
May 21, 2018
Messages
8,527
If IW and OW are two reports you can start with them. If we can get a solution for these two then others should be similar. You will need to show in a table form what data gets extracted (before and after). Also if there is related data it may be in more than one table. Example UserIDs may go into a user/client table with the transaction table having a key relating to the user table. Again this would be iterative. We can propose a solution for an IW report give it back and let you run it on real data. You will likely fine things in the real data not accounted for in the sample data.
 

CJ_London

Super Moderator
Staff member
Local time
Today, 11:14
Joined
Feb 19, 2013
Messages
16,607
just to add to my previous post

there are additional validation routines you can run in combination with begintrans and committrans commands

e.g. compare line count with the reported cheque count, total value of cheques with some other value etc

also, for similar but varying reports you might have a 'properties' table to store the 'properties'. Load this before your import and apply the appropriate value

e.g.
FileType...Event...Value
Cheques..Start...."User ID*"
Cheques..End.....""
Cheques..Validation1..."*Number of cheques*"
Cheques..Validation2..."5" 'Number of columns
BankAccount..Start...."*Opening Balance*"
BankAccount..End....."*Closing Balance*"
BankAccount..Validation1..."*Closing Balance*"
BankAccount..Validation2..."4"

use a crosstab query as your recordset and in your line of code you would have something like

if txtLine like rst!Start then

But you don't want to make the code too complex otherwise you'll end up rewriting it to cater for every new wrinkle
 

lookforsmt

Registered User.
Local time
Today, 14:14
Joined
Dec 26, 2011
Messages
672
Thanks CJ_London for the code and suggestion. I will give it a try to code, thanks.

Thanks MajP, I have tried and replicated the data on the text file as close to real report. Will try and use the earlier codes provided myself. But I might require your assistance in completing the codes.

Thanks
 

lookforsmt

Registered User.
Local time
Today, 14:14
Joined
Dec 26, 2011
Messages
672
Hi! MajP,
i tried to do exactly the way you had provided the code earlier, but then it does not give me any data. I have changed the names of the headers to the current text file.
The previous thread sample report and this one has quite dissimilarities between them. I might be putting them in the wrong place.

thanks
 

CJ_London

Super Moderator
Staff member
Local time
Today, 11:14
Joined
Feb 19, 2013
Messages
16,607
with regards my code, disable this bit temporarily

Code:
if txtLine like "USER ID*" Then 'next lines contain the data I want until line =""
            datastart=true
            Line Input #FileNum, txtLine
            lineID=lineID+1
        elseif txtline="" then 'data finished
            datastart=false
        end if

you should the see every line so you might have to change these statements

txtLine like "USER ID*"
txtline=""

you will see a | instead of a tab - and I've assumed it is a tab - it might just be spaces or some other hidden space character
 

MajP

You've got your good things, and you've got mine.
Local time
Today, 06:14
Joined
May 21, 2018
Messages
8,527
So I started to demo the import for IW, and making the code a little more modular to be able to write several different import schemes. I have specific questions on IW.
It appears there are three groups of transactions (5 for the first, 3 in the second, and 1 in the third). In between the groups there is additional header type information, which makes it challenging to determine when to start and stop import. In your example every transaction is from the same Loan Acct Number 1234567890123. Will that be the case for a specific SOL ID / User ID? This would make reading the data very easy. Once you find the first SOL ID the next Line is USER ID. After that your transactions start. The first data in the next line is the Loan Acct number. If that does not change from then on you can check if the loan acct number is in the line. If so you run the import code if not ignore.
 

lookforsmt

Registered User.
Local time
Today, 14:14
Joined
Dec 26, 2011
Messages
672
HI! Thanks MajP for helping me on this.

To clarify on your below query
It appears there are three groups of transactions (5 for the first, 3 in the second, and 1 in the third). In between the groups there is additional header type information, which makes it challenging to determine when to start and stop import.
Actually there is only two groups, the first group is continued on 2nd page with 7 records and 2nd group has 1 record.

I have changed it in the attached IW Sample file.
The change is the "COUNT : 7" is moved one line before the "Total Amount For Sol Id :"
Apologies for not providing in the first place.

In your example every transaction is from the same Loan Acct Number 1234567890123. Will that be the case for a specific SOL ID / User ID? This would make reading the data very easy.
The Loan Acct Number mentioned is example, it can be repeated under the same SOL ID or it can different Loan Acct Numbers. But in no circumstances will the same Loan Acct Number be duplicated in two SOL IDs
User ID is common for all YYYAPP

Thanks again for helping me with the code
 

Attachments

  • IW Sample.txt
    3 KB · Views: 105
  • IW_1.png
    IW_1.png
    11.3 KB · Views: 108

lookforsmt

Registered User.
Local time
Today, 14:14
Joined
Dec 26, 2011
Messages
672
Thanks CJ_London for the suggestion. I will try and understand the code. Unfortunate i not able to write the code on my own.
 

CJ_London

Super Moderator
Staff member
Local time
Today, 11:14
Joined
Feb 19, 2013
Messages
16,607
No problem - you are focusing on Maj's solution, so I'll drop out
 

MajP

You've got your good things, and you've got mine.
Local time
Today, 06:14
Joined
May 21, 2018
Messages
8,527
I think I got a set of rules that will work, but will have to code it to test. It looks a lot easier than your previous thread which had multiples MOM, Runs, and comments.

1) Start reading until you get the SOLID. Store That
2) Next line is user ID. Store That
3) Next lines are transactions until you get to a line that is empty. Not sure how to check for that but assume it is a line with just a carriage return.
4) keep reading until you get to SOL ID. If that does not change then keep reading else store it and user id.
5) Return to line 3.

I believe the table looks like
SOL_ID
User_ID
Loan_Acct_Number
Debit_Acct_Number
Accepted_Date
Amount
Start_Check_Number
Bank_Name
Presentation_Date
Number_Lodged
Set_Number
 

lookforsmt

Registered User.
Local time
Today, 14:14
Joined
Dec 26, 2011
Messages
672
Yes absolutely the headers are correct and in the same order.
It is not complicated like my previous thread. Most of the text files are somewhat similar like this (IW Sample) and hopefully i should be able to replicate them.
thank you
 

MajP

You've got your good things, and you've got mine.
Local time
Today, 06:14
Joined
May 21, 2018
Messages
8,527
So I completed the IW import leveraging the previous code. So this is the only new code
Code:
Public Sub ImportIW(strFile As String)
' ***************************************************
' * Open a Text File And Loop Through It            *
' ***************************************************
  Dim intFile As Integer
  Dim StrIn As String
  Dim TheSOL_ID As String
  Dim TheUser_ID As String
  Dim TheLoanAcctNo As String
  Dim TheDebitAcctNo As String
  Dim TheAcceptedDate As String
  Dim TheAmount As String

  Dim TheStartChqNo As String
  Dim TheBankName As String
  Dim ThePresentationDate As String
  Dim TheChqLodgedNo As String
  Dim TheSetNo As String
  Dim TempArray() As String
   
  Dim StartRead As Boolean
  intFile = FreeFile()
  Open strFile For Input As #intFile
  
  Do While Not EOF(intFile)
    Line Input #intFile, StrIn
    If InStr(StrIn, "SOL ID :") > 0 Then
       'Debug.Print strIn
       TheSOL_ID = FindAfter(StrIn, "SOL ID :")
      ' Debug.Print TheSOL_ID
    ElseIf InStr(StrIn, "USER ID :") > 0 Then
       TheUser_ID = FindAfter(StrIn, "USER ID :")
       StartRead = True
      ' Debug.Print TheUser_ID
    ElseIf StrIn = "" Then
      StartRead = False
    ElseIf StartRead Then
      TempArray = GetCleanArray(StrIn)
      TheLoanAcctNo = TempArray(0)
      TheDebitAcctNo = TempArray(1)
      TheAcceptedDate = TempArray(2)
      TheAmount = TempArray(3)
      TheStartChqNo = TempArray(4)
      TheBankName = TempArray(5)
      ThePresentationDate = TempArray(6)
      TheChqLodgedNo = TempArray(7)
      TheSetNo = TempArray(8)
     ' Debug.Print TheLoanAcctNo
     ' Debug.Print TheDebitAcctNo
     ' Debug.Print TheAcceptedDate
     ' Debug.Print TheAmount
     ' Debug.Print TheStartChqNo
     ' Debug.Print TheBankName
     ' Debug.Print ThePresentationDate
     ' Debug.Print TheChqLodgedNo
     ' Debug.Print TheSetNo
      InsertIW TheSOL_ID, TheUser_ID, TheLoanAcctNo, TheDebitAcctNo, TheAcceptedDate, TheAmount, TheStartChqNo, TheBankName, ThePresentationDate, TheChqLodgedNo, TheSetNo
    End If
  Loop
  Close #intFile
End Sub
Public Sub InsertIW(TheSOL_ID As String, TheUser_ID As String, TheLoanAcctNo As String, TheDebitAcctNo As String, TheAcceptedDate As String, TheAmount As String, TheStartChqNo As String, TheBankName As String, ThePresentationDate As String, TheChqLodgedNo As String, TheSetNo As String)
  'Put your real table name below
  On Error GoTo errlbl
  Const TableName = "tblIW"
  Dim strSql As String
  Dim rs As DAO.Recordset
  
  'Insert was just too long, figured easier to do with recordset
  'strSql = "Insert Into " & TableName & "(Sol_ID, User_ID, Loan_Acct_Number, Debit_Acct_Number, Page, MOM_ID, MOM_TXT, MOM_MMC, Department_Name) VALUES ( " & TheSegment & ", " & ThePrintDate & ", " & ThePage & ", " & TheMOM & ", " & TheMOM_Txt & ", " & TheMOM_MMC & ", " & TheDepartment & ")"
  'Debug.Print strSql
  'CurrentDb.Execute strSql
   Set rs = CurrentDb.OpenRecordset("tblIW", dbOpenDynaset)
   rs.AddNew
     rs!SOl_ID = TheSOL_ID
     rs!User_ID = TheUser_ID
     rs!Loan_Acct_Number = TheLoanAcctNo
     rs!Debit_Acct_Number = TheDebitAcctNo
     rs!Accepted_Date = CDate(TheAcceptedDate)
     rs!Amount = CCur(TheAmount)
     rs!Start_Check_Number = CLng(TheStartChqNo)
     rs!Bank_Name = TheBankName
     rs!Presentation_Date = CDate(ThePresentationDate)
     rs!Number_Lodged = CLng(TheChqLodgedNo)
     rs!set_Number = TheSetNo
   rs.Update
  Exit Sub
errlbl:
  Debug.Print Err.Number & " " & Err.Description & " " & vbCrLf & strSql
  Resume Next
End Sub

This is my imported data
Code:
SOL_ID	User_ID	Loan_Acct_Number	Debit_Acct_Number	Accepted_Date	Amount	Start_Check_Number	Bank_Name	Presentation_Date	Number_Lodged	Set_Number
1203	YYYAPP	1234567890123	1234567890	7/9/2019	$15,000.00	6110	CBDU	7/15/2019	1	C6752440
1203	YYYAPP	1234567890123	1234567890	7/9/2019	$10,000.00	6111	HBSC	11/15/2019	1	C6752441
1203	YYYAPP	1234567890123	1234567890	7/9/2019	$9,000.00	6112	CBXD	7/11/2020	1	C6752442
1203	YYYAPP	1234567890123	1234567890	7/9/2019	$115,000.00	142960	EDMB	8/1/2019	1	C6752443
1203	YYYAPP	1234567890123	1234567890	7/9/2019	$18,000.00	142962	CBDU	12/11/2019	1	C6752444
1203	YYYAPP	1234567890123	1234567890	7/9/2019	$10,000.00	350	CBDU	7/19/2019	1	C6752445
1203	YYYAPP	1234567890123	1234567890	7/9/2019	$10,000.00	6117	HBSC	11/15/2019	1	C6752446
1203	YYYAPP	1234567890123	1234567890	7/9/2019	$12,000.00	351	CBDU	7/19/2019	1	C6752450
 

Attachments

  • Read Multiple Reports.accdb
    836 KB · Views: 99

Users who are viewing this thread

Top Bottom