Ability to communicate

[Pat Hartman]

I put this thread here, not because it is in any way political, but because it may offend some people. This is your trigger warning.

I have a client who is having a technical issue and has not been able to use the word automation part of my application since October. Their division was sold and from May-Oct the technical teams from both companies worked to transfer the applications to the new company servers. The final piece went into place in Oct and that was when my app stopped working. The clients called me and I determined that there was nothing I could do to fix the problem since the problem was environmental. They have been using Citrix for 4 years so I took a chance and installed the app so it would run from their LAN. It worked fine but was a little sluggish since the BE database is now 1500 miles away from them rather than in the basement. So, I left this option available on ONE computer so they could print letters if they needed to but Citrix is faster and also allows them to work from home so I don't want to go back to strictly LAN based operation. So, FOUR months ago, I was able to narrow the problem down to Citrix, the office version running on Citrix (2013 vs O365 on local desktop), some random firewall that is interfering with the Word automation. some Office security setting that is interfering with Word automation. The app gets as far as opening the Word doc but fails with 5460 or 5981 when the code tries to populate the first bookmark.

I have no capability to modify anything in their environment so testing to get to this level was even a challenge but I was working mostly with the "from" team. Technically, the users are not out of operation. They are just majorly inconvenienced and if they get any more customers, one person won't be able to handle the printing.

So for four months I have been communicating with their tech people trying to get them to try things. What happens if you install O2013 on a desktop, does the app break, what happens if you install O365 on citrix, does the app work? What EXACTLY changed in Oct??? Blah, blah, blah. So far nothing - and they have never contacted one of the users with a "try this". I finally suggested to the client that he try to start a case with Microsoft to see if he could get someone with Office knowledge to see if they could figure out what broke. That meeting happened yesterday. It went on for an excruciating 3.5 hours and we accomplished absolutely nothing. There were EIGHT people on the call plus myself. Only three of the nine people spoke English. The two users and me. The MS rep's accent was somewhat understandable if we could get him to speak slowly and enunciate. I think he understood that the problem was between Access and Word but only on Citrix. The other 6 people who represented the technical support team for the client's new parent company were unintelligible. If I could understand two words in a sentence, it was a success but that is just too many blanks to fill in for a technical conversation. They also did not understand me when I spoke so it wasn't just my old and failing ears. The issue was bi-directional. I am not at all surprised that in four months, they have made no progress with resolving this problem. Their tech support is a tower of Babel. Given the names in the email, I'm guessing we were dealing with at least 3 languages + English. I have written out our testing attempts in excruciating detail as we did them since the techs may have less trouble with written English than with verbal to no avail.

Here is a silly example of our attempts to do something. They have the app set up to run in isolation which means, you can't access any other windows app via that connection so I wanted to run the app using a desktop connection. They got that and remembered they had made one for me in October. Then I tried to get them to copy the batch file we use to run the app from the server to the local PC. Nobody knew were it was. So I told them that the batch file is executed each time the user logs in so if they would look at the Citrix script that runs when a user logs in, they would see its name and path. That took fully 15 minutes before I could get someone, anyone, to locate the Citrix script, open it and find the name and location of the bat file. My first attempt was to get them to copy the file manually from the server folder to the Citrix desktop but they didn't understand what I was asking for. Then the bat file wouldn't work because the user account couldn't MD. So I tried to get them to open the script in EDIT mode -stop double clicking on it - and delete the first two lines. Then change the last two lines to reference a local folder so that all we were left with was - copy the FE from the server to a local folder. Run it. That was another 15 minutes. It didn't work, which was not unexpected. But then I tried to get them to look at the settings of that desktop to see if there was something that would interfere with automating work. Like open word and check out the settings. Even the MS guy couldn't understand that.

I have two questions. The first is pretty much rhetorical.
1. Why do companies hire people who are required to communicate with English being the common language when their grasp of English is so poor that they can barely have a "hi, how are you" conversation? Are employers afraid to be called racists if they say English is a requirement of the job? I have no idea whether any of these guys were technically competent and frankly, I don't care. They were completely ineffective because we couldn't communicate. It doesn't matter how cheap it was to hire these people, they were not earning whatever token wage they were being paid.
2. Any bright ideas? Types of tests I can get them to try?

 

[The_Doc_Man]

As to your question #2, I'm going to presume that you looked up those two errors (which frequently occur together.)

https://exalt.zendesk.com/hc/en-us/articles/218489527-Error-5981-5460-Application-defined-or-Object-defined-error-when-creating-Word-document

Beyond that, there are such things as intelligent firewalls that not only can intercept traffic between points A and B, but also can tell which protocol is in use AND what program originated the use. I ran into this using CDO from Access. It wasn't that SMTP was disallowed, but that it was coming from Access rather than Outlook. So the question would be to ask about their app-specific firewall filters. (Maybe.)

As to your trailing question #1, it is because outsourcing appears at first glance to be cheap; further, the employment issues following COVID and people who want to work from home have only exacerbated the problem of not having enough local talent to go around. In a sense, certain types of high-tech talent ARE frangible - but if the supply of replacements is too low, you run out of "spare parts" and are stuck with what is still on the market.

 

[Pat Hartman]

Thanks for the link. I never thought to look for both errors together. That certainly looks like what might be causing the problem. If we hadn't had such impossible language issues, they IT people could have envisioned something like that also because it was exactly what I was trying to describe to them. The MS guy should have jumped on this immediately as the solution if he really was an Office "expert".

The only thing is - was this available for 2013 and why would they have enabled it on Citrix but not on the desktops? But, it is still a contender so I sent it off to the folks on the call. two months ago I sent them 4 similar articles but not one that mentioned the specific property that this one refers to.

The Citrix people just love to make tunnels to apps because it is more "secure" than creating a desktop for them but it is flat out impossible to debug anything when all you have access to is the compiled app. You can't see anything else like trusted locations.

 

[The_Doc_Man]

You might want to ask them if their firewalls retain their logs for a while, or if they are exported. If you are getting a rejection from the firewall and know the date/time of that rejection, it should be easy to track down.

 

[Pat Hartman]

The rejection seems to be coming from Word, if you believe the messages.

 

[The_Doc_Man]

When I had the problem with CDO, it gave me messages from Access but I was also able to work with the IT security group to identify that their firewall was rejecting the connection, thus triggering the "connection rejected" messages. You have multiple elements working together and it would perhaps be helpful to see if the firewall is taking any part in this. If the fix in the article helps, great. But debugging at task-interaction level can be complex since every task downstream from the connection attempt will be relaying something. Might not hurt to look at the system event logs, either. I'm not saying it IS the firewall. I'm merely suggesting that if something doesn't pan out right away, it might help to look to other links in the chain of network devices and tasks that are taking part in the connection.

 

[Pat Hartman]

I have asked them on multiple occasions to check the firewalls and see what they were rejecting. You keep assuming that these people understand what I am asking them to do. I don't know enough about how any of this security works so that I can tell them exactly what they need to look at. They are supposed to know how the firewalls work. They are supposed to know what the logs will tell them. That is their job. The best I can do is to tell them what the application is trying to do. We'll see if they take any action on the link you sent on Monday.

 

[The_Doc_Man]

Roger that, and it sounds like you have a REAL "winner" of an IT support group on your hands.

Try to identify the IP address or the MAC address of the workstation trying to send the message. Hint: Use the command prompt ARP -a command. Find the approximate time of day of a mail failure event. Ask them to filter the log files to only show the physical or IP addresses you find. The physical address in question will be the most frequently identified address in the ARP tables. And firewall logs WILL be in chronological order by default. Good firewalls will allow you to filter like that. Cheap-arse firewalls? Not so much.

It might help you to know if they have a "domain map" (might also be called a "network schematic") that shows what machines in are in each group of machines that share the same router or subnet - a.k.a. in the same "network zone". If you can locate where your failing stations are located and where their targeted network partners are located in that map, then you can identify the specific routers/firewalls that your connection must cross. That will tell you which firewalls/routers should have the log you need if such logs exist. And of course, it IS possible that they DON'T have such a map, but if they don't, it tells you WHY you have trouble getting answers.

Unfortunately, it MAY be that their problem is that they use a lot of simpler routers when security concerns would have warranted use of firewalls. In that case, the logs you need might not exist. BUT if you can identify the relevant zones and the boundary machines that isolate each zone, you can at least point an accusing finger at fewer machines.

 

[Pat Hartman]

Unfortunately, it MAY be that their problem is that they use a lot of simpler routers when security concerns would have warranted use of firewalls. In that case, the logs you need might not exist. BUT if you can identify the relevant zones and the boundary machines that isolate each zone, you can at least point an accusing finger at fewer machines.

You're wasting your time Doc. I don't have the bandwidth to learn their tools and their jobs. It is totally impossible without a model of their environment on which to experiment or the basic knowledge to even know what a properly configured, secure installation of Citrix would look like. The client is simply going to have to escalate this through his management.

 

[The_Doc_Man]

I guess I was spoiled by the fact that when we had our issue with the Navy, we had network maps to identify zones and figure out inter-boundary routes. All I have left, Pat, is to wish you good luck with this problem - and hope that the article I found for you was helpful enough.