I have a pdf file, now I want to process natural language with artificial intelligence similar to chatgpt in Access and with VBA coding, so that when the user asks a question, he can go and find the answer from inside the pdf and display the best answer. Please guide me for this, what algorithm and what libraries should I use in VBA.
AI can mean a lot of things to a lot of people. I don't want to make assumptions about whether your AI needs to be a genius level or average level or moron level. However, whatever level you want, getting there will mostly be up to you. I suspect that the AI part of your code will be the most convoluted, and when it comes to AI, "native" Access is dumber than a box of rocks. The AI is going to depend on your NI - native intelligence - because Access by itself is very limited. So you will be providing most of that code yourself.
While what you want is certainly possible, VBA is not a fast-executing language, so extensive AI computations might be slower than you would expect. Further, Access has size limitations for modules, so expect to do a lot of compartmentalization. So far as I know and based on what I have read in this forum in the past couple of years, there IS no AI library for Access from Microsoft, though if you found a 3rd party library compatible with VBA's method of calling library modules, it should be possible to use it. I am not aware of any major 3rd party libraries either, but some of my colleagues might have heard of one. They will have to step in if they know of such a thing.
You would probably need to learn about application objects, because there are PDF libraries that get loaded if you have the Adobe products. You would need to open the PDF via an application object methodology by having a reference to the Adobe library. You would then perform that actual file opening step using routines in the library. To use some 3rd party AI library, again I would suspect that you would need a reference to this hypothetical library and would have to create the intellect object to get it to do things.
One last warning: When exposing your AI to users, make sure you have lots of safeguards and input checks built-in, because of something I learned long ago: Artificial Intelligence cannot cope with natural stupidity.
The following code was suggested by CHATGPT. Please help in writing the codes of LoadTrainedModel - PreprocessText and LoadAnswers functions and designing the model.
Sub TensorflowSearch()
' Load the trained model
Dim model As Object
Set model = LoadTrainedModel()
' Prompt the user for a search query
Dim searchQuery As String
searchQuery = InputBox("Enter a search query:")
' Process the search query
Dim processedQuery As String
processedQuery = PreprocessText(searchQuery)
' Load the available answers from a data source (e.g., Access table)
Dim answers As Variant
answers = LoadAnswers()
' Find the best answer using cosine similarity
Dim bestAnswer As String
bestAnswer = FindBestAnswer(processedQuery, answers, model)
' Display the best answer
MsgBox bestAnswer
End Sub
Function LoadTrainedModel() As Object
' Load the trained model
' Code to load trained model goes here
End Function
Function PreprocessText(text As String) As String
' Preprocess the text
' Code to preprocess the text goes here
End Function
Function LoadAnswers() As Variant
' Load the available answers from a data source (e.g., Access table)
' Code to load answers goes here
End Function
Function FindBestAnswer(query As String, answers As Variant, model As Object) As String
' Find the best answer using cosine similarity
' Code to calculate cosine similarity and find the best answer goes here
End Function
can you provide an example of a question that might be asked and the sort of answers you would consider to be acceptable. Also is this 'any old pdf' or a specific one that is always used - or at least has a consistent format
I use a .exe called pdftotext which can extract the text from a pdf. I use this for automating the import of pdf invoices into an accounting system.
It may be too simple for your needs but if of interest, you can download the .exe from here
First, be aware that ChatGPT is known (not just suspected) to have fabricated methodology when you ask it for code and designs. We have examples in this forum where someone asked for it to write a VBA module and what it produced wouldn't run. If the syntax errors were corrected, it also would not have produced the requested results. Therefore, with due respect to the Open AI folks, take that code only with extreme care.
This sounds like you want to hire someone. The members of this forum often go out of their way to help people, but to put it in perspective, you have asked a "big ask." This is a project potentially big enough to hire experts or persuade a hobbyist - and if you CAN get someone to do it for free, better be prepared to offer royalties on its use. I don't know if you realize just how complex a problem you offered. I'm retired so I would have some time on my hands, but I know enough about AI that I wouldn't touch this problem with a ten-foot pole.
Your "pre-process text" function is, in itself, guaranteed to become a finite automaton, which is a whole field of study in and of itself. You will have to parse text carefully because finite automatons can become non-finite with one minor mistake in any text-based state transition.
"Tensorflow" and "Load Trained Model" are probably functions of Python and machine learning. You might be able to code VBA in Visual Studio Code with GitHub Copilot as the AI source.
But incorporating Access into the mix is probably unnecessary.
With the investigation done, the possibility of extracting the right answer from within a pdf is very difficult due to specific formats. I know that this is a big project, but with the precious guidance of dear professor, I can take action and complete my personal plan, not a project. I want to change my question as follows: I have created a table called Tab_Quality, which contains a field called "Description" and 4 records are recorded in this table as follows:
1- Patient safety means careful monitoring and control of risks and accidents and reporting errors in medical centers.
2- National accreditation based on quality and efficiency and quality dimensions is done annually to evaluate medical centers with an emphasis on improving quality and patient safety.
3- National accreditation is carried out with the goals of quality, justice and the implementation of patient safety standards.
4- Patient safety is done with the goals of reducing risks and reporting errors and analyzing them in the hospital.
Now I want to use VBA coding and artificial intelligence design to check the "Description" field and return the result to the user when a question is received in the txt_srch textbox.
For example, what does the validation phrase mean when the user in txt_srch? type the program to display record 2.
And also what are the goals of national accreditation when the user types? Record 2 feedback
Also, when the question is asked, explain to me about patient safety? Give feedback on all 4 options
The purpose of the program is to process natural language and choose the best answer.
Why do you think that a database developement system like MS Access, which is 30 years old and hasn't undergone any major updates apart from the interface and a few small things, is suitable for this?
Such a task could be outsourced from Access to third party services though. Like:
Access gathers info => sends info as request
Third party service receives info => processes info => sends a response
Access receives processed info => displays results
Such a task could be outsourced from Access to third party services though. Like:
Access gathers info => sends info as request
Third party service receives info => processes info => sends a response
Access receives processed info => displays results
You are now delving into interrogatory logic, which is the study of the relationship of an answer to its corresponding question. It is used as a means to validate that SQL is responding properly to a query. The problem of deciding whether SQL returned data conformant to an input query with a vocabulary of fifty to one hundred keywords is already a project at the level you could sell as a product.
When considering the vocabulary associated with natural language questions, this is a highly advanced subject. You went to ChatGPT and only got a vague layout. For you to ask us to help you with this is flattering - because it appears you think we are going to be able to help you with a subject that could easily develop into a master's thesis or doctoral dissertation in the field of computer intelligence. But I have to suggest that it would be rare - not impossible, but rare - to find that level of help on this forum. To have Access do this may well be beyond its abilities. With all due respect, this may be the wrong venue for you. A database forum seems to be an ill match for the kinds of things you propose to do.
Please visit the link if you're able on your end. It's part of the documentation of OpenAI's chat completion technology. I assume it would be similar to what ChatGPT would respond, but with the added benefit that you can choose models and even fine-tune them. The role of MS Access in this case is to make requests using one of the multiple HTTP libraries available from the references list.
As for what it will send, that is up to the requirement. For instance, if you need to send the contents of a PDF file, then you can either use another service that can extract entities from the PDF file, use some third party software, or perhaps Acrobat's library. Truth is, as I mentioned, you can outsource those things. Access, being a database system will be able to gather information from different sources, then it can make requests to one of the endpoints of the REST API to process that information and receive responses to use later. I disagree that it would be necessary to reinvent the wheel and do a lot of research in that area, there are plenty of AI services available that you can use for decision making and if there is none, there are Python libraries available for that as well, like Spacy.
Please visit the link if you're able on your end. It's part of the documentation of OpenAI's chat completion technology. I assume it would be similar to what ChatGPT would respond, but with the added benefit that you can choose models and even fine-tune them. The role of MS Access in this case is to make requests using one of the multiple HTTP libraries available from the references list.
As for what it will send, that is up to the requirement. For instance, if you need to send the contents of a PDF file, then you can either use another service that can extract entities from the PDF file, use some third party software, or perhaps Acrobat's library. Truth is, as I mentioned, you can outsource those things. Access, being a database system will be able to gather information from different sources, then it can make requests to one of the endpoints of the REST API to process that information and receive responses to use later. I disagree that it would be necessary to reinvent the wheel and do a lot of research in that area, there are plenty of AI services available that you can use for decision making and if there is none, there are Python libraries available for that as well, like Spacy.
I was hoping you'd attempt an explanation as to what your intention is. Referring us to documentation doesn't really explain what YOU plan to do with it.
Apologies, George. I thought no further explanation was necessary given the context of my previous responses. But if I'm not being clear, here's the plan:
In post #8, I am suggesting OP to use OpenAI's GPT 3.5 service. In that same post, I'm asking OP to clarify what exactly he wants to do with the PDF file, because it was not clear to me. He posts in #9 that he still needs the AI, but the requirement has become more complex. Other respondents claimed that Access is a bad tool to solve this. I agree to an extent. However, I remain firm in my belief that Access can orchestrate the operations by outsourcing the processing of what it doesn't know how to do to a third party, allowing you to not switch to an unfamiliar platform, potentially cutting costs.
2. I suggest using the aforementioned service because, with little effort, one can receive chat completions. Now, you might be wondering why chat completions are a good candidate for solving OP's problem. First, because he wants to use something similar to ChatGPT. Second, because that service in particular allows you to fine-tune a model to better suit your needs. Third, because it's highly available, easy to use and it has been proven effective, hence its popularity. Fourth and most important, because OP expects people to write open questions on his textbox, potentially containing zero matches if you simply carry out a similarity algorithm, thus requiring NLP technology, something that can extract the meaning of the question and try to match that to the contents of the 4 records OP mentioned in post #9.
3. What is the plan? There is a form. This form contains a textbox where a user can perform questions about a patient. There is also a textbox where the answer will appear.
4. Let's say the user types in this textbox: "what does the validation phrase mean?". Said user then triggers an event. Let's say it's a click of a button, which is next to the textbox and has a caption "Answer me". The on click event of the button is triggered. This private subroutine has the necessary code to make a request to OpenAI's chat completion endpoint, such as:
Code:
Private Sub cmdAnswerMe_Click()
RequestAnswer
End Sub
5. Let's define RequestAnswer. According to OP, he will have 4 records in a table, these records have a Description field. So, depending on how this table is designed, we have to open a recordset where these 4 records can be retrieved to be compared to the text content of the textbox. So, what we must send could potentially be arranged like this string:
Code:
Private Function BuildSentence(str As String) As String
'code to open a recordset and fills a rstCount and a rstSentences variables using it
BuildSentence = _
"Which of these " & rstCount & " sentences answers this question better: '" & _
str & "'" & rstSentences & ". " & _
"Please explain why, briefly."
End Function
Code:
Public Function SendRequest(strUrl As String, strApiKey As String, strQuestion) As String
Dim oRequest As Object
Set oRequest = CreateObject("MSXML2.ServerXMLHTTP.3.0")
With oRequest
.Open "POST", strUrl
.SetRequestHeader "Authorization", "Bearer " & strApiKey
.Send strQuestion
.WaitForResponse
SendRequest = .responseText
End With
End Function
Code:
Private Sub RequestAnswer()
Dim response As String
Dim theEndpoint As String: 'define it
Dim theApiKey As String: 'define it
Dim theQuestion As String: theQuestion = BuildSentence(Me.txt_srch)
If Not IsNull(Me.txt_srch) Then
response = SendRequest(theEndpoint, theApiKey, theQuestion)
Me.txtResponse.value = response
' check the format, of course
End If
End Sub
And with just a few lines of code, that .accdb file now integrates AI. This untested code should display in the response textbox the answer "National accreditation based on quality and efficiency and quality dimensions is done annually to evaluate medical centers with an emphasis on improving quality and patient safety."
