Tag Info

Hot answers tagged

15

No, you cannot use rsync to transfer files to Amazon. It uses its own protocol for the service. But you can access their storage via third-party services, like www.s3rsync.com, then you'll be able to use rsync, and your data will be finally transfered to S3 storage. Or you can use special utilities designed for S3 storage. There are: s3sync, s3command, ...


7

Why a seperate file manager when there is s3fs, which makes it possible to mount S3 buckets via fuse? See http://code.google.com/p/s3fs/wiki/FuseOverAmazon for installation instructions. That way you won't notice any difference between local files and your S3 bucket in nautilus.


5

Can you give us a rough idea of your bandwidth / storage requirements? Generally the cheapest cost is to purchase your own server and then colocate it at an inexpensive datacenter. But that is more work to manage it plus more cost up-front to buy the servers. Edit: Ok, that's about $3000-$4000/mo in Amazon S3 costs. 25TB/mo could probably be handled ...


5

Yes, you are right, you need to install fusefs-s3fs, once you have done that, make sure you put fusefs_enable="YES" somewhere in your /etc/rc.conf (so that fuse will start after a reboot). Now create a file in /etc/ called passwd-s3fs and set its perms to 640. In that file add your S3 access-key and secret key as follows: accessKeyId:secretAccessKey on ...


4

Amazon's EC2 service is far more complex and powerful than a simple VPS. Under EC2 you can manage one or more "instances", with the ability to automate the starting/stopping/creation/destruction of instances. Each instance is a virtual machine running the operating system of your choice, like with a VPS. EC2 is not oriented towards disk space (although the ...


4

If your site is running 24x7 you should not use a on demand instance, on demand instances are for temporary jobs that you need a machine for (like load balancing for extra traffic). What you want is a heavy utilization reserved instance. You pre-pay a one time upfront fee to get a discounted hourly rate on the instance over the lifespan on the contract. ...


3

S3 buckets can be accessed by anyone as long as you know: the bucket name the access key the secret key There are many tools that allow you to connect to an S3 bucket and up/download files, including: S3 browser Cyberduck s3fs (CLI) s3cmd (CLI) … I'm sure a web search for S3 clients will deliver more results.


3

Your Dropbox is encrypted during transmission and while stored on their servers. Dropbox themselves do, however, have access to there files but it is against their policy to look at your files (note that it's against policy though technically possible). If you're concerned with the security of your files on any of these services, the solution is to encrypt ...


3

Amazon provides their own CLI tools now too. From http://aws.amazon.com/cli/ Using familiar syntax, you can view the contents of your S3 buckets in a directory-based listing. $ aws s3 ls s3://mybucket LastWriteTime Length Name ------------- ------ ---- PRE myfolder/ 2013-09-03 10:00:00 1234 ...


3

Jungle Disk Workgroup Edition fits your requirements. There's even an iPhone/iPad app.


2

It depends on what risks you are trying to mitigate Ensure you can always access the data, even if Amazon goes down, stops providing the service, etc. For this, you need a local backup. Keep in mind that you will get charged for moving the data into S3 in the first place, and again for getting it back out. Incremental backups are an advantage here. ...


2

Yes, you may use Bucket Policies to configure bucket in a way that all existing and new files will be publicly available. Here is the bucket policy you need to apply: { "Version": "2008-10-17", "Statement": [ { "Sid": "AllowPublicRead", "Effect": "Allow", "Principal": { "AWS": "*" }, "Action": [ ...


2

Panic's Transmit app can do it on the Mac. You need to transfer files. Maybe to an FTP or SFTP server, or the cloud via Amazon S3, or using WebDAV. You maintain a website, do backups, or upload photos. You need Transmit, a Mac OS X FTP client.


2

It all depends on just how sensitive your information is. If you're looking for real security for your files, encrypt them before you upload them. As others have already mentioned, Dropbox encrypts anything that isn't in your public folder for you with AES using a 256-bit password, which would take longer than the age of the universe to crack by brute ...


2

Without knowing what it is you're trying to serve (sold media? can it be compressed?) I'll offer up a few different suggestions. You could use Coral and a mod_rewrite - It's free and would offload a lot of the bandwidth. Or another commercial CDN such as CacheFly (they work with small businesses almost exclusively), Akamai, or Limelight Networks There's ...


2

I agree with davr. The cheapest alternative is to get your own dedicated server. Take a look at this for instance: http://www.server4you.net/root-server/server-details.php?products=5 Hosting in Germany is relatively cheap (compared to US prices). This is a dedicated Server with 2x 1TB HDD, QuadCore CPU and 8GB Ram. Traffic is 100Mbit/s flat. All in all 99 ...


2

You might want to look at Duplicity. It produces encrypted archives which it can upload to a variety of storage types, among them Amazon S3. It does incremental uploads: Duplicity uses a standard backup scheme in the sense that for the first backup, duplicity performs a full backup. Additional backups are stored in separate files and record only ...


2

If all you are doing is scraping websites, I would suggest an alternate solution: Do not use Amazon EC2. EC2 requires a skill level beyond simply being able to administer a unix-style operating system. EC2 also assumes that you can handle situations by yourself, there is no one to bail you out. I understand that currently EC2 is free to new signups for ...


2

TL;DR CrossFTP is in the Ubuntu app store, you must "buy" it for the purchase price of $0.00. So an Ubuntu One account is required, to install that way. For a mountable file-system try S3QL, which is available through a PPA. Okay, it seems part of the confusion with s3fs is that there are THREE of them with mostly the same name with varying levels of ...


2

Click the triangle next to "Other Computers". (Or if you're using Arq 3, hover your mouse over "Other Backup Sets" and click the "Show" link). There you'll find your old backup data. Select it and click the "Adopt This Backup Set". This will cause Arq to adopt the settings and backup data it was using on your old computer, so you don't have to re-upload. ...


2

A one liner: diff <(awk '{print $4}' s3 | sort) <(awk '{print $1}' local | sort) Basically, what it does is compare the output of awk, which we'll use to print the fourth and first column of the files, respectively. We need to sort the output in case the hashes are in different orders. The <(…) syntax is process substitution and allows us to run ...


2

The public data sets are not hosted on Amazon S3 as such, rather they are provided as Amazon Elastic Block Store (EBS) snapshots. While these are stored on S3 in fact, it is not possible to access such a snapshot directly, rather you need to create a new EBS volume from it and attach it to an Amazon EC2 instance for further processing at your discretion. ...


2

This is caused by backups to iCloud, Apple's cloud storage service. While I can't find any official information from Apple, some sites are reporting that Apple is using both Azure and S3 to store iCloud data. I've not found any way to limit the bandwidth through the device itself; limiting the throughput on the router seems to be the only way to throttle ...


1

Under the assumption that you have created an IAM user with a dedicated policy it might miss the s3:ListAllMyBuckets action. If so, you can browse a single bucket in Cyberduck by connecting to mybucket.s3.amazonaws.com instead of the default S3 hostname. Refer to http://trac.cyberduck.ch/wiki/help/en/howto/s3#Externalbuckets.


1

After some time I was able to develop a code in bash which check the md5sum from both, s3 and my local files and remove the local files that are already in amazon s3: #!/bin/bash datacenter="amazon" hostname=`hostname`; path="backup/server245" s3=`s3cmd ls --list-md5 -H s3://company-backup/company/"$datacenter"/"$hostname"/"$path"/` s3_list=`echo ...


1

If all people were logged in as the same user, then you have no chance to find out, who it was, because for the computer, everybody is the same because of the same user-account they used. I would highly recommend you, to set different users for these people. Maybe you have a chance to recover your file. You could use for example TestDisk. You should use ...


1

Seeing nothing in the AWS Console, I suspected that the ec2-import-volume command from the EC2 API tools may be the culprit and found a related ec2-describe-conversion-tasks. Executing this showed two pending and two expired tasks that apparently have been attempting conversion of disk image data that I'd since deleted and doing so over and over again. ...


1

Amazon offers a free tier for the micro, so you will find just using a micro will cost you nothing other than going above the data transfer (iirc, it's 15GB) Otherwise, the hourly rate is exactly what it is, if you run your instances 24x7 you will be charged 744 hours a month. If you normally ran 2 servers to host your site/service and during offpeak you ...


1

I think this will answer your question: Scroll down and start reading at Michael Fisher's answer. DO NOT hardcode your key on the client. The right way to do this is (via a secure channel) to deliver a pre-built request with an encrypted signature -- this signature can be built with the API provided by S3. You can extract the ...


1

I share your worry about trapping data in proprietary formats, but I don't really see how that is relevant for a backup service. The premise in backup is that you have a copy of the data - in that case, it's easy to switch service by simply dumping the other service. The data isn't trapped as long as you have it. The same applies for synchronisation ...



Only top voted, non community-wiki answers of a minimum length are eligible