The tag has no usage guidance.

learn more… | top users | synonyms

0
votes
0answers
20 views

Should I go for Hadoop?

I hope discussing scenarios and solutions isn't out of context, if it is then please suggest. Details: Windows server 2012 R2 with IIS and MS SQL standard edition, using C# in the front end with Ajax ...
0
votes
0answers
8 views

to pass the acton name,start date and end date and status to another action in oozie workflow

I have a requirement like i need to pass an actions output(like action_name,success/failed) to another action in a oozie workflow. Kindly pass some information how to schedule the actions ? This is ...
0
votes
0answers
8 views

Creating a cluster on Ambari

I have installed ambari and the server is running - as can be seen by the screenshot. What is the procedure to then create a cluster? The documentation literally says "follow the onscreen ...
0
votes
0answers
26 views

Hadoop Security Flaw Detection Code [closed]

I need to do white-hat hacking of a Hadoop cluster that is just using out of the box security. I'm looking for actual code or very specific procedures. These should produce very clear results, like ...
0
votes
0answers
13 views

How does ResourceManager communicates to NameNode

Does ResourceManager talks to NameNode when running an application in YARN. If Yes, would like to know when and how.
0
votes
0answers
9 views

Is HDFS concurrent reading less efficient than reading a replicated copy?

If I have two identical files A and B on HDFS, and then have two simultaneous read tasks on those files. Is scenario 1 faster? Scenario 1: read A + read B Scenario 2: read A + read A If the ...
0
votes
1answer
26 views

Will Hadoop benefit from just one disk?

If I use a single node setup with Hadoop where it has only one disk, I can easily use it to play around with Hadoop and stuff, but will there actually be any benefit to using Hadoop in this case over ...
1
vote
1answer
75 views

Open eclipse with a linux user that doesn't have graphical environment (created from console)

I created an user from console in my Ubuntu Destkop 14.04 LTS, doing this: sudo addgroup hadoop sudo adduser --ingroup hadoop hduser I used that user for doing all sort of stuff, because i'm using ...
0
votes
0answers
38 views

Why would a dead datanode be working on a Hadoop program?

I have 3 datanodes hadoop01, hadoop02, and hadoop03. When I run hadoop dfsadmin -report, I get that 1 data node is dead and the offending datanode (Last contact: Mon Mar 16 22:49:47 CDT 2015) is ...
0
votes
0answers
39 views

How to Deploy an Hadoop MapR Sandbox in The Cloud?

I'm New to Cloud computing so give me a favor if its a silly question. I have a OVA Template of MapR Sandbox downloaded from this below link. ...
0
votes
0answers
51 views

How do I find which disk (volume) is failing?

I am using stock Apache Hadoop 1.1.1 and I can't get a datanode to start due to: 2015-04-23 09:12:48,138 INFO org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from ...
0
votes
0answers
51 views

Setup multi node cluster in ubuntu 12.04 using Cloudera?

I setup five virtual machines (Ubuntu 12.04 LTS ) using Virtual box, then I configured ssh password less authentications from master system to all slave systems. I have installed Cloudera CDH 5.3 in ...
0
votes
0answers
82 views

Cannot access Ambari Namenode WebUI

I am trying to setup the Ambari HDP using Vagrant. I can access localhost:8080 fine with all services up and running. But for some reason I cannot access localhost:50070 to get to the Namenode WebUI. ...
0
votes
1answer
30 views

Hadoop output during run — What is the number right after the timestamp

When Hadoop runs, it puts out the messages we've all seen many times. E.g.: 2015-03-30 10:09:08,750 [main] WARN org.apache.hadoop.conf.Configuration - dfs.data.dir is deprecated. Instead, use ...
0
votes
0answers
922 views

ERROR: Can't get Master address from zookeeper; znode data == null

I have two linux machine one is master machine(192.168.8.174) and another one is slave1 machine(192.168.8.173). I have installed and configured Hadoop 2.6.0 in fully distributed mode successfully. ...
0
votes
0answers
64 views

How to delete directories older than x days in a folder in Hadoop

How to delete directories older than x days in a folder in Hadoop. The directories are having date qualifier at end. Like X_Y_Z_YYYYMMDD. I have to compare the date of the directory with the provided ...
0
votes
1answer
61 views

Copy data from local disk to hdfs in a federation of HDFS

I am trying to configure a Federation of HDFS. Therefore, I configured the $ cat etc/hadoop/hdfs-site.xml in both hosts (hadoop-coc-1, and hadoop-coc-2). The problem is that with this configuration, ...
0
votes
1answer
15 views

What does “streaming-off-disk” mean?

I am taking a course offered by www.statistics.com—Advanced Analytics and Machine Learning with Hadoop—and I reading through course material. I stumbled upon the following statement (emphasis mine): ...
0
votes
1answer
78 views

Is it normal behavior for the Reduce tasks to wait until the Map tasks finish?

I have a 50 GB MapReduce job which is (basically) a WordCount app and it has the following Map/Reduce job percentages (listed at the end of the question). It seems like the reducers want to wait until ...
1
vote
0answers
16 views

View all CURRENT parameters for a hadoop installation?

I have inherited a hadoop installation and I would like to query all usage parameters and configuration options. I am sifting through the conf directory and it is helping, but this is an old ...
1
vote
1answer
99 views

Hadoop Uninstall : Shall I remove linux-image packages?

I removed hadoop from my machine via apt-get remove --purge hadoop In the output I see the following packages were not removed : The following packages were automatically installed and are no longer ...
0
votes
0answers
31 views

Adding additional harddrive to hdfs

The new hard drive was partitioned and mounted to /data. mode of /data was changed to 755 and the ownership was changed to hadoop user. The following configuration was added to hdfs-site.xml at the ...
0
votes
1answer
187 views

Permission denied (publickey) error when using scp

I had created 3 linux instances on the Amazon EC2 cloud without a passphrase for the private key. I had stored the private key in a file id_rsa in one of my instances(the master) to allow it to ssh ...
-2
votes
1answer
51 views

Can't download hadoop

Am I taking crazy pills or is it impossible to download hadoop right now? I am looking at http://www.apache.org/dyn/closer.cgi/hadoop/common/ to find mirrors for it and absolutely none of them work ...
0
votes
0answers
33 views

Emails from root while working with Horton Sandbox

I recently installed the VM for Horton Sandbox (CentOS 64) and I consistently get the notification: You have new mail in /var/spool/mail/root and when I go there I see notices of the form: 22 ...
0
votes
0answers
91 views

How to run Hadoop MR job in Normal privilege

I have installed Hadoop 2.3.0 on Windows8 and able to execute MR jobs successfully. But when I trying to execute MR jobs in normal privilege (without admin privilege) means job get fails with ...
1
vote
0answers
129 views

adding nodes to hadoop cluster virtualbox

I am trying to setup a mini hadoop cluster with virtualbox: hosts file (hosts file properly configured): hadoop@rui-VirtualBox:/home/rui$ cat /etc/hosts | grep 10. 10.0.13.96 hnode1 10.0.13.97 ...
0
votes
0answers
10 views

Issue in configuring address for resource manager in hadoop 2.4

Ok here's all that have been accomplished. I have created entry in hosts file, and they are legitimate and correct(verified by pinging and also telnet-ing). But after I configure the yarn-site.xml to ...
0
votes
0answers
24 views

HDFS with small block size

Does anybody run HDFS with small block size in production? Under "small" I mean just like usual FS. Say, 4Kb. While it looks like going against main HDFS principles, I think it may be a good solution ...
0
votes
2answers
398 views

Unable to SSH, cannot create /dev/null/.ssh

I am trying to deploy hadoop with a master and slave node. But the master fails to start tasttracker and datanode processes in slaves, as it is not able to ssh into slaves. The error message is 'Could ...
0
votes
1answer
564 views

Disable RAID Controller at BOOT and Partition Each Disk

I have some decent HP Proliants server that come with "HP Smart Array P410i Controller" enabled, I am using these boxes to set up a Hadoop cluster and I know, RAID is for sure a no-no for Hadoop since ...
1
vote
1answer
341 views

Sqoop JDBC connection timeout after import but before hive import

We are using Sqoop v1.4.4 14/05/27 13:49:14 INFO sqoop.Sqoop: Running Sqoop version: 1.4.4-cdh5.0.0 Sqoop 1.4.4-cdh5.0.0 git commit id 8e266e052e423af592871e2dfe09d54c03f6a0e8 When I'm importing a ...
1
vote
0answers
720 views

Hadoop - java.io.IOException: Connection reset by peer when creating when creating a new directory

I have installed hadoop 2.4.0 as a single node for learning purposes but after I start hadoop and create a directory using the command: hadoop fs -mkdir /tmp I get the following error: ls: Failed ...
0
votes
4answers
776 views

Mac OS X 10.9 Mavericks breaks passwordless SSH to localhost (for hadoop)

Sorry if this sounds like a repeat, but I'm fairly sure it's specifically Mac OS X 10.9 related, it works fine for me on my 10.8 machine. I've done everything identically, as usual: $:~ ssh-keygen -t ...
1
vote
0answers
119 views

hadoop cluster high availability error couldn't find or load main class

[hduser@mn1 bin]$ ./hdfs zkfc -formatZK Error: Could not find or load main class ”-Djava.library.path=”.home.hduser.2.3.0″.lib” I did every step just like in this tutorial, TWICE. The error still ...
0
votes
1answer
99 views

How to start after installing OpenStack

I have installed OpenStack following the tutorial from here I want to know that how can I deploy simple PHP application using Apache Hadoop on OpenStack? Any reference?
2
votes
2answers
328 views

Unable to send files via SCP

I tried using the scp command with the -i option to transfer the file from local machine to remote EC2 instance: Akhis-Macbook-Pro:~ aswinakhilesh$ sudo scp -i Mykey.pem FileA ...
0
votes
1answer
331 views

Disable Hadoop Symlinks on Windows

I'm running Hadoop 2.2.0 on Windows 7. When I try to execute my job, Hadoop tries to create symlinks in the temp folder on my local file system. This is not working as I do not have the necessary ...
0
votes
0answers
470 views

How to install Mahout on Windows 8.1?

My goal is to build up a recommendation system and after going through many articles, I came across Mahout as a simple, yet effective way to go on. I already have XAMPP installed on my system. How ...
0
votes
1answer
623 views

How to remove large number of files/folders in linux

We are using hadoop to split a table into smaller files to feed to mahout, but in the process, we created a huge amount of _temporary logs. we have an nfs mount for the hadoop volume so we can use ...
0
votes
2answers
7k views

How to move files to hdfs in hadoop using unix command

I have 2 directories "dft" and "hdfs" in the unpacked Hadoop folder. I am trying to copy the file(StartUnit.txt) in dft into hdfs. The corresponding command and exception are as shown below ...
0
votes
0answers
291 views

Failed to set permissions of path: ..\taskTracker\cyg_server to 0700

While running a job in Cygwin in Hadoop cluster, the permission failed. $ bin/hadoop jar hadoop-examples-*.jar wordcount inputwords outputwords2 Patch for HADOOP-7682: Instantiating workaround file ...
2
votes
2answers
5k views

Slave: ssh: connect to host slave port 22: Connection timed out

i have done with single node cluster on two different machine,I have made one as master(192.168.1.1) and other m/c as slave(192.168.1.2), I am successfully able to ping between two machine,I have made ...
1
vote
2answers
1k views

wants to remove cdh4(hadoop) from ubuntu 12.04

i have installed cdh3 pseudo mode in my ubuntu 12.04(precise) system using the following web reference cdh3-install-ubuntu12.04 now i tried to install cdh4 to replace cdh3 but,unfortunately found ...
0
votes
1answer
142 views

Does formatting the HDFS filesystem remove my data?

I'm starting to learn Hadoop. If I want to format the HDFS filesystem I need to execute bin/hadoop namenode -format Does it mean that it will format my filesystem, losing everything I have there? ...
1
vote
0answers
735 views

Why does eclipse report an error when I use it under a hadoop account?

I created two accounts, one is a common user, the other is just for hadoop, which is a superuser as root. I can load eclipse normally under my common user account, but I could not use it under hadoop ...
0
votes
1answer
141 views

Hue not loading in browser

I have a Hue installation on my local machine and the browser loads it very well in the browser when I give the URL 10.20.30.40:8888 in my browser. When i do the same for a remote machine (I ...
1
vote
1answer
1k views

Pig tutorial in local mode returns OutOfMemoryError

I am new with pig and I try to learn from pig tutorial on Apache website. I am working with hadoop 1.0.1 and PigVersion 0.11.1. In the tutorial, it is proposed to try pig on two examples (called ...
1
vote
0answers
500 views

Cygwin ssh-host-config on windows for hadoop

I'm new to cygwin and hadoop, please anyone could explain which mirror link is the proper to install cygwin what version of hadoop is bug free ( .sh works good ) is it must to run below commands ...
0
votes
1answer
352 views

Passwordless ssh on cloudera hadoop

I am using cdh4 on ec2. On the master node I don't see any keys pre-built and in fact passwordless ssh from master to hadoop slave node(s) does not work out-of-the-box. [ec2-user@ip-10-196-blah-blah ...