SSH SFTP, using Curl (1 Viewer)

namliam

The Mailman - AWF VIP
Local time
Today, 13:56
Joined
Aug 11, 2003
Messages
11,696
OK, so we use Curl to do a load of FTP.

Now I have been assigned with making the first ever SSH connection, but I am seriously struggling getting it up and running.

I having this command line and error issue...
Code:
C:\Curl-7\Bin\url --key D:\Keys\PrivateKey.ppk SFTP://IPADDRESS:22 -u Username:Password -v
* About to connect() to IPADDRESS port 22 (#0)
*   Trying IPADDRESS... connected
* Connected to IPADDRESS (IPADDRESS) port 22 (#0)
* Failed to read known hosts from C:\Documents and Settings\User\Application Data/_ssh/known_hosts
* SSH host check: 2, key: <none>
* Closing connection #0
* SSL peer certificate or SSH remote key was not OK
curl: (51) SSL peer certificate or SSH remote key was not OK
It seems to not be recognizing the host, which is ok as I have not set the 'right' host.

Anyone have any clue(s) to help me along?
 

Banana

split with a cherry atop.
Local time
Today, 05:56
Joined
Sep 1, 2005
Messages
6,318
I'm familiar with SSH but not cURL. Looking at cURL quickly, it seems to build atop of other protocols, of which SCP/SFTP is one of them.

Therefore, I'd suggest that you first validate that you can get SSH to work without cURL. That way we can troubleshoot the issue with credential more effectively.

Since you're using Windows, you need to get a SSH client. PuTTY can do the job just fine and if you are comfortable using command-line, I prefer to use Plink (which is same thing as PuTTY, only without a GUI and can be either downloaded on its own or as a bundle with the rest of PuTTY).

Next, I noticed you're using both password and key. I could be wrong but AFAIK, it's an either/or proposition - you either use a password or a key but not both. The SSH will authenticate based on the sshd_config file, which enumerates allowed authenticate methods and possibly the order as well, so that may be also reason why you can't get authenticated because it's reading the wrong method first and thus refusing your login request.

Do you know what the remote server's sshd_config says/expects for the authentication?

When you connect to a server for first time, you usually are asked basically, "Do you trust this server?" and if answered in affirmative, an entry is made into known_hosts file. That entry is then consulted in future. It is possible that cURL or SFTP is configured to not even prompt that choice to you and thus refuse to proceed any further. A separate attempt via SSH may help you create the necessary information in the known_hosts for it to work.

To run a SSH test with a plink, try that:

Using password authentication:
Code:
C:\...\plink -h IPADDRESS -u Username -v

Using public authentication:
Code:
C:\...\plink -h IPADDRESS -u Username -i C:\...\...ppk -v

(Last one is a guess assuming that plink follows the ssh closely in the parameters -- it should but just in case it isn't kosher, check the manual on plink/PuTTY for correct syntax.)


HTH.
 

namliam

The Mailman - AWF VIP
Local time
Today, 13:56
Joined
Aug 11, 2003
Messages
11,696
I probably should have mentioned this, using PSFTP / Putty, I have no problems (manually) connecting with this:

Code:
psftp -l Username -pw Password -i Keyfile.ppk IPADDRESS:22 -v

I can even put files there and take them off (manually)
However using PLink, I seem to run into some trouble too...
Command:
Code:
plink -P 22 -l User -pw Password -i KeyFile.ppk IPADDRESS -v
Code:
Offered public key
Offer of public key accepted
Authenticating with public key "rsa-key"
Access granted
Opened channel for session
Allocated pty (ospeed 38400bps, ispeed 38400bps)
Started a shell/command
Last login: Fri Apr  2 16:36:04 2010 from myIP
Server sent command exit status 1
Disconnected: All channels closed
Connecting just after this with PSFTP works again fine... I cant seem to find the error in my login for PLink either....

Next, I noticed you're using both password and key
I was confused about this too, but it requires it to authenticate my (private) key?? It is the Pass phrase for the key that I generated.

Do you know what the remote server's sshd_config says/expects for the authentication?
Nope and no way I am getting that neither... support from the 'other side' is VERY minimal ATM :(

Code:
When you connect to a server for first time, you usually are asked basically, "Do you trust this server?"
Yes got that and (eventually) saved it as trusted after confirming it was right.
Do you know where this "Known_hosts" is stored?
It seems like its looking for it in that folder but the _ssh folder dont exist??

Strangely enough it seems to mix Windows "\" and unix "/" in the path?

FYI
PSFTP without key, fails to connect "Disconnected: Unable to authenticate"
Without Password, it simply prompts for a passphrase...
 

Banana

split with a cherry atop.
Local time
Today, 05:56
Joined
Sep 1, 2005
Messages
6,318
I probably should have mentioned this, using PSFTP / Putty, I have no problems (manually) connecting with this:

Okay, it's probably cURL insisting on a known hosts or something like that. At least we know that it's not the creds that's the problem. We may want to look at the known_hosts file then.

I was confused about this too, but it requires it to authenticate my (private) key?? It is the Pass phrase for the key that I generated.

OH! I totally forgotten you could encrypt your private key with a passphrase. Now that makes sense. So ignore that. Sorry.

Yes got that and (eventually) saved it as trusted after confirming it was right.
Do you know where this "Known_hosts" is stored?
It seems like its looking for it in that folder but the _ssh folder dont exist??

For plink, you mean?

I looked quickly but it sounds like it doesn't use known_hosts file? I don't have a plink handy right now so I can't check it quickly but if it's not in PuTTY's folder then I guess the alternative is to either get different client, write one yourself by hand (it should be just text file but it has to be formatted in a particular way though) .

Strangely enough it seems to mix Windows "\" and unix "/" in the path?

I've not noticed that before but I do know that for MySQL on Windows, they do the same thing -- their configuration files path use Unix "/" even on a Windows machine because MySQL will interpret Windows "\" as escape character so it must be either one of the format:

Code:
C:/Program Files/MySQL/MySQL Server 5.1/my.cnf

or

Code:
C:\\Program Files\\MySQL\\MySQL Server 5.1\\my.cnf

I'd bet PuTTY has to do something similar because SSH is actually a UNIX tool running atop of Windows. In that case, best to follow along.

HTH.
 

namliam

The Mailman - AWF VIP
Local time
Today, 13:56
Joined
Aug 11, 2003
Messages
11,696
Okay, it's probably cURL insisting on a known hosts or something like that. At least we know that it's not the creds that's the problem. We may want to look at the known_hosts file then.
I can understand it is trying to find said file in said folder, but...
know_hosts no extension?
_ssh doesnt exist yet, I can create it, but if Curl needs it, I would expect it to be there already?

What value(s) and/or format does it need to be?

I've not noticed that before but I do know that for MySQL on Windows, they do the same thing -- their configuration files path use Unix "/" even on a Windows machine because MySQL will interpret Windows "\" as escape character so it must be either one of the format:

Code:
C:/Program Files/MySQL/MySQL Server 5.1/my.cnf

or

Code:
C:\\Program Files\\MySQL\\MySQL Server 5.1\\my.cnf

I'd bet PuTTY has to do something similar because SSH is actually a UNIX tool running atop of Windows. In that case, best to follow along.

HTH.
perculiar behaviour, but the mix of / and \ are there,its probably a case of a messed up error vs an actual bug... but it looks funky ...
 

Banana

split with a cherry atop.
Local time
Today, 05:56
Joined
Sep 1, 2005
Messages
6,318
No file extensions. SSH comes from UNIX which doesn't use file extensions but it's just a plain text file.

Looking at my known_hosts, the format should be:

Code:
[IP-Address or hostname] [encryption] [private key value]

all on one long line. For each hosts, it gets its own line. Example:

Code:
192.168.0.100 ssh-rsa XXXXXXX....
192.168.0.200 ssh-rsa XXXXXXX....


But I should caution there are different formats (I seem to recall PuTTY using its own format though it can accept SSH's format but you have to convert it or something like that. Been too long, though.)



Hmm, didn't realize it's mixed / and \. If there are spaces in the filename, I"d expect / to escape the spaces, though.

Example path for my Mac OS X folder:
Code:
/Users/banana/Documents/My\ Cool\ Stuff

would be like this in Windows (roughly):
Code:
"C:\Users\banana\Documents\My Cool Stuff"

I hope that makes sense...
 

Banana

split with a cherry atop.
Local time
Today, 05:56
Joined
Sep 1, 2005
Messages
6,318
Oops, forgot one more answer to your other question: Yes, known_hosts are automatically created by SSH when you successfully log in to a unknown hosts. Normally, you should get a prompt by SSH "unknown host. Do you want to add the hosts?" or something similar... that should then create the known_hosts. Did you get that when you used the SSH?

On UNIX, the file would go to ~/.ssh/known_hosts. On Windows, it depends on the ssh client you use.
 

namliam

The Mailman - AWF VIP
Local time
Today, 13:56
Joined
Aug 11, 2003
Messages
11,696
Normally, you should get a prompt by SSH "unknown host. Do you want to add the hosts?" or something similar... that should then create the known_hosts. Did you get that when you used the SSH?
No question unfortunatly :( if it had this problem probably wouldnt exist :(

On UNIX, the file would go to ~/.ssh/known_hosts. On Windows, it depends on the ssh client you use.
Yes, well ~ is My Documents\UserName, which in itself could pose some challenges. I cannot find any setting/switch/etc to change this path either.

Possibly if I go tinkering with the registery... but erm...


Code:
[IP-Address or hostname] [encryption] [private key value]
Yes I tried this in just about every itteration I can think of, including or excluding spaces/columns where applicable again in every possible combination I can think off, still no joy however :(
 

Banana

split with a cherry atop.
Local time
Today, 05:56
Joined
Sep 1, 2005
Messages
6,318
No question unfortunatly :( if it had this problem probably wouldnt exist :(

If you're not asked that, then it may be that it's already created but cURL can't find that file. Have you searched your hard drive for that file?

Yes, well ~ is My Documents\UserName, which in itself could pose some challenges. I cannot find any setting/switch/etc to change this path either.

If that's where cURL expects the working directory to be, then you should be able to find a .ssh folder in that folder... Is it there? (It may be hidden as well)

Possibly if I go tinkering with the registery... but erm...

Understandable. I don't like that idea myself.
 

namliam

The Mailman - AWF VIP
Local time
Today, 13:56
Joined
Aug 11, 2003
Messages
11,696
If that's where cURL expects the working directory to be, then you should be able to find a .ssh folder in that folder... Is it there? (It may be hidden as well)
I thought of it being hidden, not the case. I dont understand though if the "question has already been asked", which it has by PuTTY... and the host should already exist, then how can it be looking for the Known_hosts???

I have done a search for the filename, but none found, baring the one I created :(

This thing has be completely caught in a loop.
 

michael21

New member
Local time
Today, 05:56
Joined
Apr 28, 2010
Messages
1
Namliam, I'm having the same problem and was wondering if you found a fix for this?
 

namliam

The Mailman - AWF VIP
Local time
Today, 13:56
Joined
Aug 11, 2003
Messages
11,696
My 'fix' for now anyways is to have it scripted using PSFTP and have for now moved on to bigger and badder things :(

However we need to revisit this 'someday' to fix the temp scripted solution, we will see how "permanent" this temporary solution turns out to be.
 

willycan

New member
Local time
Today, 08:56
Joined
Mar 16, 2018
Messages
1
Better late then never? So I got it to work and felt this forum should have the correct answer. FYI had to modify the links for the software as this is my first post.

I use the Viktor Szakats binary of curl and my setup is as follows:
Download the Win64 - Generic Win64 x86_64 Viktor Szakats binary (should be compressed in a 7z file) from http curl.haxx.se/download [dot] html
Extract all the files into the above directory, make sure they are not inside another directory
Download and Install vcredist_x64.exe (MS Visual Studio .NET 2008 Redistributable tools) found on http [www] microsoft [dot] com/en-us/download/details.aspx?id=15336
Download and Install Win64OpenSSL_Light from http slproweb.com/products/Win32OpenSSL [dot] html (Make sure to install it to c:\Program Files\OpenSSL\)
Add the working directory to the path statement in Windows by:
System Properties
Advanced
Environment Variables
Path (cut and paste this line at the end of it)

;C:\Program Files\cURL;C:\Program Files\OpenSSL;C:\Program Files (x86)\GnuWin32\bin;

The key file has to be in a system accessible location along with the known_hosts file (best to put it in the same directory)
You can put it in your User Profile directory, but the system directory is better if you want other users or other applications to run a script
The known_hosts file is the most difficult setup item. Steps to setup known_hosts:
1. Navigate to C:\Windows\system32\config\systemprofile\AppData\Roaming
2. Create a directory called _ssh
3. Navigate to the _ssh directory and create known_hosts text file
4. On a linux based machine issue the command$ ssh-keyscan -t rsa -p 22 sftp.address.com username
4 a. where -p 22 represents the port number, and sftp.address.com represents the ip or domain address of the host ()
5. Copy the output starting with the host or ipaddress in square brackets followed by the port and ssh-rsa AAAA...
5 a. Example [192.168.1.1]:22 ssh-rsa AAA ...
6. Paste the copied text into the known_hosts text file you created earlier, with one line per host

So the finished known_hosts file should be in the path
C:\Windows\system32\config\systemprofile\AppData\Roaming\_ssh\known_hosts
 

Users who are viewing this thread

Top Bottom