The tag has no wiki summary.

learn more… | top users | synonyms

0
votes
0answers
58 views

ERROR: Can't get Master address from zookeeper; znode data == null

I have two linux machine one is master machine(192.168.8.174) and another one is slave1 machine(192.168.8.173). I have installed and configured Hadoop 2.6.0 in fully distributed mode successfully. ...
0
votes
0answers
21 views

Why do i get request timedout while pinging a system?How to ping successfully? [closed]

I have added an Hadoop-vm-appliance in Vmplayer5.1.After powering on VM 192.168.74.... is shown and i start hadoop through command -> ./start-hadoop and i login. Since i implement MapReduce for my ...
0
votes
0answers
22 views

How to delete directories older than x days in a folder in Hadoop

How to delete directories older than x days in a folder in Hadoop. The directories are having date qualifier at end. Like X_Y_Z_YYYYMMDD. I have to compare the date of the directory with the provided ...
0
votes
1answer
12 views

Copy data from local disk to hdfs in a federation of HDFS

I am trying to configure a Federation of HDFS. Therefore, I configured the $ cat etc/hadoop/hdfs-site.xml in both hosts (hadoop-coc-1, and hadoop-coc-2). The problem is that with this configuration, ...
0
votes
1answer
13 views

What does “streaming-off-disk” mean?

I am taking a course offered by www.statistics.com—Advanced Analytics and Machine Learning with Hadoop—and I reading through course material. I stumbled upon the following statement (emphasis mine): ...
0
votes
1answer
24 views

Is it normal behavior for the Reduce tasks to wait until the Map tasks finish?

I have a 50 GB MapReduce job which is (basically) a WordCount app and it has the following Map/Reduce job percentages (listed at the end of the question). It seems like the reducers want to wait until ...
1
vote
0answers
11 views

View all CURRENT parameters for a hadoop installation?

I have inherited a hadoop installation and I would like to query all usage parameters and configuration options. I am sifting through the conf directory and it is helping, but this is an old ...
1
vote
1answer
42 views

Hadoop Uninstall : Shall I remove linux-image packages?

I removed hadoop from my machine via apt-get remove --purge hadoop In the output I see the following packages were not removed : The following packages were automatically installed and are no longer ...
0
votes
0answers
17 views

Adding additional harddrive to hdfs

The new hard drive was partitioned and mounted to /data. mode of /data was changed to 755 and the ownership was changed to hadoop user. The following configuration was added to hdfs-site.xml at the ...
0
votes
1answer
154 views

Permission denied (publickey) error when using scp

I had created 3 linux instances on the Amazon EC2 cloud without a passphrase for the private key. I had stored the private key in a file id_rsa in one of my instances(the master) to allow it to ssh ...
-2
votes
1answer
43 views

Can't download hadoop

Am I taking crazy pills or is it impossible to download hadoop right now? I am looking at http://www.apache.org/dyn/closer.cgi/hadoop/common/ to find mirrors for it and absolutely none of them work ...
0
votes
0answers
25 views

Emails from root while working with Horton Sandbox

I recently installed the VM for Horton Sandbox (CentOS 64) and I consistently get the notification: You have new mail in /var/spool/mail/root and when I go there I see notices of the form: 22 ...
0
votes
0answers
59 views

How to run Hadoop MR job in Normal privilege

I have installed Hadoop 2.3.0 on Windows8 and able to execute MR jobs successfully. But when I trying to execute MR jobs in normal privilege (without admin privilege) means job get fails with ...
0
votes
0answers
86 views

adding nodes to hadoop cluster virtualbox

I am trying to setup a mini hadoop cluster with virtualbox: hosts file (hosts file properly configured): hadoop@rui-VirtualBox:/home/rui$ cat /etc/hosts | grep 10. 10.0.13.96 hnode1 10.0.13.97 ...
0
votes
0answers
9 views

Issue in configuring address for resource manager in hadoop 2.4

Ok here's all that have been accomplished. I have created entry in hosts file, and they are legitimate and correct(verified by pinging and also telnet-ing). But after I configure the yarn-site.xml to ...
0
votes
0answers
22 views

HDFS with small block size

Does anybody run HDFS with small block size in production? Under "small" I mean just like usual FS. Say, 4Kb. While it looks like going against main HDFS principles, I think it may be a good solution ...
0
votes
0answers
51 views

Shared drive with VMware Player

I downloaded Hortonworks Sandbox2.1 and VMware Player and I want to create a file in windows that I can use to access data with Hadoop via VMware Player. First I tried enabling the shared folder and ...
0
votes
0answers
30 views

Distributing master key in hadoop

For the master node passwordless-ly ssh into the slaves, the master needs to distribute its ssh keys to the slaves. Copying keys using ssh-copy-id asks for the user password. If there are hundreds of ...
0
votes
2answers
253 views

Unable to SSH, cannot create /dev/null/.ssh

I am trying to deploy hadoop with a master and slave node. But the master fails to start tasttracker and datanode processes in slaves, as it is not able to ssh into slaves. The error message is 'Could ...
0
votes
1answer
240 views

Disable RAID Controller at BOOT and Partition Each Disk

I have some decent HP Proliants server that come with "HP Smart Array P410i Controller" enabled, I am using these boxes to set up a Hadoop cluster and I know, RAID is for sure a no-no for Hadoop since ...
1
vote
1answer
258 views

Sqoop JDBC connection timeout after import but before hive import

We are using Sqoop v1.4.4 14/05/27 13:49:14 INFO sqoop.Sqoop: Running Sqoop version: 1.4.4-cdh5.0.0 Sqoop 1.4.4-cdh5.0.0 git commit id 8e266e052e423af592871e2dfe09d54c03f6a0e8 When I'm importing a ...
1
vote
0answers
572 views

Hadoop - java.io.IOException: Connection reset by peer when creating when creating a new directory

I have installed hadoop 2.4.0 as a single node for learning purposes but after I start hadoop and create a directory using the command: hadoop fs -mkdir /tmp I get the following error: ls: Failed ...
0
votes
3answers
646 views

Mac OS X 10.9 Mavericks breaks passwordless SSH to localhost (for hadoop)

Sorry if this sounds like a repeat, but I'm fairly sure it's specifically Mac OS X 10.9 related, it works fine for me on my 10.8 machine. I've done everything identically, as usual: $:~ ssh-keygen -t ...
1
vote
0answers
103 views

hadoop cluster high availability error couldn't find or load main class

[hduser@mn1 bin]$ ./hdfs zkfc -formatZK Error: Could not find or load main class ”-Djava.library.path=”.home.hduser.2.3.0″.lib” I did every step just like in this tutorial, TWICE. The error still ...
0
votes
1answer
68 views

How to start after installing OpenStack

I have installed OpenStack following the tutorial from here I want to know that how can I deploy simple PHP application using Apache Hadoop on OpenStack? Any reference?
1
vote
2answers
225 views

Unable to send files via SCP

I tried using the scp command with the -i option to transfer the file from local machine to remote EC2 instance: Akhis-Macbook-Pro:~ aswinakhilesh$ sudo scp -i Mykey.pem FileA ...
0
votes
1answer
220 views

Disable Hadoop Symlinks on Windows

I'm running Hadoop 2.2.0 on Windows 7. When I try to execute my job, Hadoop tries to create symlinks in the temp folder on my local file system. This is not working as I do not have the necessary ...
0
votes
0answers
371 views

How to install Mahout on Windows 8.1?

My goal is to build up a recommendation system and after going through many articles, I came across Mahout as a simple, yet effective way to go on. I already have XAMPP installed on my system. How ...
0
votes
1answer
512 views

How to remove large number of files/folders in linux

We are using hadoop to split a table into smaller files to feed to mahout, but in the process, we created a huge amount of _temporary logs. we have an nfs mount for the hadoop volume so we can use ...
0
votes
2answers
5k views

How to move files to hdfs in hadoop using unix command

I have 2 directories "dft" and "hdfs" in the unpacked Hadoop folder. I am trying to copy the file(StartUnit.txt) in dft into hdfs. The corresponding command and exception are as shown below ...
0
votes
0answers
260 views

Failed to set permissions of path: ..\taskTracker\cyg_server to 0700

While running a job in Cygwin in Hadoop cluster, the permission failed. $ bin/hadoop jar hadoop-examples-*.jar wordcount inputwords outputwords2 Patch for HADOOP-7682: Instantiating workaround file ...
2
votes
2answers
4k views

Slave: ssh: connect to host slave port 22: Connection timed out

i have done with single node cluster on two different machine,I have made one as master(192.168.1.1) and other m/c as slave(192.168.1.2), I am successfully able to ping between two machine,I have made ...
1
vote
2answers
1k views

wants to remove cdh4(hadoop) from ubuntu 12.04

i have installed cdh3 pseudo mode in my ubuntu 12.04(precise) system using the following web reference cdh3-install-ubuntu12.04 now i tried to install cdh4 to replace cdh3 but,unfortunately found ...
0
votes
1answer
102 views

Does formatting the HDFS filesystem remove my data?

I'm starting to learn Hadoop. If I want to format the HDFS filesystem I need to execute bin/hadoop namenode -format Does it mean that it will format my filesystem, losing everything I have there? ...
1
vote
0answers
624 views

Why does eclipse report an error when I use it under a hadoop account?

I created two accounts, one is a common user, the other is just for hadoop, which is a superuser as root. I can load eclipse normally under my common user account, but I could not use it under hadoop ...
0
votes
1answer
114 views

Hue not loading in browser

I have a Hue installation on my local machine and the browser loads it very well in the browser when I give the URL 10.20.30.40:8888 in my browser. When i do the same for a remote machine (I ...
1
vote
1answer
948 views

Pig tutorial in local mode returns OutOfMemoryError

I am new with pig and I try to learn from pig tutorial on Apache website. I am working with hadoop 1.0.1 and PigVersion 0.11.1. In the tutorial, it is proposed to try pig on two examples (called ...
1
vote
0answers
416 views

Cygwin ssh-host-config on windows for hadoop

I'm new to cygwin and hadoop, please anyone could explain which mirror link is the proper to install cygwin what version of hadoop is bug free ( .sh works good ) is it must to run below commands ...
0
votes
1answer
306 views

Passwordless ssh on cloudera hadoop

I am using cdh4 on ec2. On the master node I don't see any keys pre-built and in fact passwordless ssh from master to hadoop slave node(s) does not work out-of-the-box. [ec2-user@ip-10-196-blah-blah ...
-1
votes
0answers
84 views

How to speed up mahout learning process? [closed]

I have been using a dataset with million documents as a training set. It takes a day for mahout to learn in my machine with 8GB ram, intel i5 2.50GHZ? Is there any tweak to improve the learning ...
1
vote
1answer
277 views

Check Oozie status in UNIX

Hadoop Admin Utility Created a basic hadoop admin utility which checks dfs health and oozie status. Checking oozie status In my above utility, I am using the below command : bin/oozie admin ...
-1
votes
2answers
560 views

Does Hortonworks Data Platform support Ubuntu or not?

I am new to the Hortonworks Data Platform. Can anyone tell me whether Ubuntu supports Hortonworks Data Platform or not?
1
vote
2answers
850 views

Setting Hostname as IP on Linux for Hadoop VM

How can I have an Ubuntu server VM set /etc/hostname/ to the value fo the VM's assigned IP address automatically on startup? I'm creating an Ubuntu server VM image to run Hadoop. When a client ...
0
votes
1answer
341 views

How can I access the ganglia web interface with ssh tunneling to monitor my EMR job?

Ive been using the standard hadoop monitoring tools with: ssh -L 9100:localhost:9100 -L 9101:localhost:9101 -o ServerAliveInterval=10 -o StrictHostKeyChecking=no -i key.pem ...
1
vote
1answer
1k views

Hadoop installation on ubuntu

I installed Hadoop on ubuntu from the .deb package when I run start-all.sh I get this error I configured the core-site.xml for hdfs:localhost:9001 but i gives me the error. I think the problem in the ...
0
votes
0answers
105 views

IDE versus SCSI Disk When Running Cloudera Hadoop VMWare CDH 4.2.0

I downloaded the Cloudera CDH 4.2.0 Hadoop image and tried to run it on the latest VMWare player for Windows 7 64-bit. My hard drives are both SSDs. When I did I saw the following pop-up. Can ...
0
votes
1answer
693 views

Configuring hadoop results in “/bin/java : no such file or directory”

I am trying to install hadoop on Cygwin. I have set JAVA_HOME as follows: \cygdrive\c\work\java\jdk1.6.0_30 in the file conf/hadoop-env.sh. Performing a echo $JAVA_HOME also displays correctly ...
2
votes
1answer
1k views

JAVA_HOME not set properly

I am trying to configure hadoop in cygwin. I have set JAVA_HOME as /cygdrive/c/work/java/jdk1.6.0_30 If I echo $JAVA_HOME, it displays correctly. If I run the command bin/hadoop version it is giving ...
2
votes
1answer
2k views

Error when Starting CYGWin SSH daemon on windows 7 home edition

I am using Windows 7 Home edition, trying to install CYGWin SSH daemon. Successfully done the same, but when I run the same, I see error in C:\cygwin\var\log\sshd.log file Privilege separation user ...
1
vote
1answer
453 views

Modify LD_LIBRARY_PATH and CLASSPATH on cluster node before running a hadoop job

I need to set LD_LIBRARY_PATH and CLASSPATH before running a job a cluster. In LD_LIBRARY_PATH i need to add location of some jars which are required while running the job, As these jars are ...