With assistance from the devs of SparkleShare, found the issue (and am waiting on an official fix while implementing a workaround below).
~\AppData\Roamin\sparkleshare\debug_log.txt is the following:
11:57:53 | Controller | ssh-agent started, PID=6972
11:57:54 | Auth | Imported key 'C:\Users\wmyers\AppData\Roaming\sparkleshare\2012-09-20 11h20.key'
11:57:54 | Auth | The following keys may be used:
2048 5f:8c:51:d7:53:d1:61:98:f6:bd:68:56:cc:ce:d2:d0 C:\Users\wmyers\AppData\Roaming\sparkleshare\2012-09-20 11h20.key (RSA)
11:59:18 | Fetcher | C:\Users\wmyers\SparkleShare\.tmp\media | Fetching folder: ssh://firstname.lastname@example.org:8110/home/storage/media
11:59:18 | Auth | Fetching host key for 22.214.171.124
11:59:20 | Auth | Skipping fingerprint check
11:59:20 | Auth | Accepted host key for 126.96.36.199
11:59:20 | Fetcher | Failed creating fingerprint: Index was outside the bounds of the array. at SparkleLib.SparkleFetcherBase.GetFingerprint(String public_key)
11:59:21 | Cmd | .tmp | git clone --progress --no-checkout --depth=1 "ssh://email@example.com:8110/home/storage/media" "C:\Users\wmyers\SparkleShare\.tmp\media"
The key line is the one at 11:59:20 with "Fetcher" at the front.
There is a fix officially being released shortly that addresses proper importation of the server's public SSH key. The current workaround is to manually copy the contents of the server's
~\.ssh\known_hosts on the client machine.
If that does not work for you, you need to wait for the next release after 0.9.2.
Also, apparently this is only related to running ssh on a non-standard port.