Episode #125 of the Stack Overflow podcast is here. We talk Tilde Club and mechanical keyboards. Listen now

Questions tagged [distributed-computing]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0
votes
0answers
6 views

How zookeeper cluster guarantee messages global order

Environment: one ZK Cluster, it constains one Leader(L) and some Followers(F1,F2...). Situation1: Client send operateA and operateB to F1, F1 transfer them to L, because of the queue between F1 ...
1
vote
0answers
36 views

I'm following the tutorial for setting up tf_jobs as given in Kubeflow documentation and I run into error in ks apply stage

I'm following this instruction for setting up TF JOBs. These are instructions that work just fine: CNN_JOB_NAME=mycnnjob VERSION=v0.4.0 This is my init code: ks init ${CNN_JOB_NAME} cd ${...
0
votes
0answers
13 views

R through Spark

I have an R shiny application, which calls and saves data from and to a database, respectively. Is it possible for time performance issues, to execute plain R code in a Spark cluster? I wish to know,...
0
votes
0answers
10 views

Is there a reliable method to do a census of nodes in a distributed network?

I'm planning on implementing a distributed system and need a census function of active nodes. I'm assuming this method deals with and mitigates Sybil attacks and tolerates malicious nodes. Is there ...
1
vote
0answers
74 views

MATLAB GPU-enabled function? [closed]

Is there a way for me to find out whether the functions I'm using in a MATLAB script are GPU-enabled? There does not seem to be a comprehensive list (though see this link for a list of image ...
1
vote
1answer
28 views

What is the correct way to turn a second computer into a simple expansion for the first? [closed]

I have two computers side-by side in the same large case. One is a gaming rig, one is currently just managing case fans and doesn't even have an OS. I have a truly insane number of peripherals. ...
1
vote
0answers
26 views

How to parallel windows pc in same network?

I have some windows pc in my local network, each of them have 8 cores cpu. I have program that takes lot of cpu resource, so I want to parallel it on all of the pc. Right now I running my program ...
-1
votes
1answer
32 views

In Linux, is concept of Process similar to Container (in microservices?) [closed]

Process is executing an instance of an application. Every process got its State, ID, address space etc, and isolated to each other. It looks similar to Container that runs a specific service? In some ...
0
votes
1answer
33 views

Running a scripted program between a rapsberry pi and a laptop

What I have: A laptop(CPU: i7 6500U, RAM:8GB, OS: Windows 8.1) and a RPi Zero What I know: Distributed computing is something that requires a lot of time and effort using special software such as ...
0
votes
1answer
2k views

Is it possible to use 2 Mac minis to do a distributed computing system

I need some advice on connecting and using 2 Mac minis as a distributed computing system. The situation is that, my current Mac mini went bad on discrete graphics card and is almost usable only in ...
0
votes
0answers
276 views

Using multiple computers as a processing and data storage cluster

I have about 17 or so old computers that by themselves can't do much, but I was wondering if there was any way to combine them so that they could run multiple VMs in a distributed way. For example, ...
0
votes
0answers
341 views

Can we use Ceph or Gluster FS to avoid IO bottleneck to start many VMs?

AIM: I want to start 50s of VM on s single server or on a set of PCs. Scenerio: I tried to boot 15 VM on a single server with enough computing power(processor) and RAM but disk is a bottleneck in ...
0
votes
1answer
466 views

running a simulation software on a remote server

some simulation softwares take so much time and processing power to finish the job. How can I run these kind of softwares on a remote server? (e.g. a finite element method software such as ANSYS, CST ...
1
vote
1answer
588 views

Delete logs after consumption: logstash

I have logstash watching a directory on the host. Every time it sees a log that matches the path I specified in the logstash config it starts to import the data into my elasticsearch cluster. Does ...
2
votes
0answers
52 views

how to keep a distributed software up to date on Linux machines

I have my own distributed software running on few RasPi (20+) in different geographical locations, but every time i change something in my app, i have to update the excuteble in all Pi's manually, ...
1
vote
1answer
905 views

Difference between “network RAM” and “distributed shared memory” (DSM)

What is the difference between "network RAM" and "distributed shared memory" (DSM)? Basically, I know that both of them provide a shared storage on RAM among all the systems in a cluster. So, what is ...
0
votes
1answer
106 views

How large can a VMware datastore be? Name of VMWare software that monitors single server/multiple servers?

I had an interview and was asked the following questions about VMWare. I'm eager to know the answers, please help me : How large can a VMware datastore be? What is the name of the VMware software ...
1
vote
1answer
632 views

How does distributed computing work?

I heard a 3D modeller say that when he renders a scene he uses multiple machines to do it because simply the hardware is not powerful enough and he is using the collective computational power of more ...
0
votes
0answers
103 views

Distributed data storage for slow network with high node churn?

What distributed data storage shall I use for reliably storing data over set of nodes with poor networking between them? I want something that: Does not expect that all nodes are reachable now, ...
1
vote
4answers
1k views

Problems when loop over a series of ssh-ed commands

I have a series of server machines which I want to run the same command on. Each command takes hours and (even though I am running the commands using nohup and setting them to run in the background) ...
0
votes
1answer
302 views

fastest non volatile memory type available for High Performance Computing/ Cloud computing? [closed]

I am working on parallel computing and distributed programs and noticed that file i/o is quite a bottleneck. It seems quite clear this particular aspect is major issue in many high performance ...
0
votes
0answers
59 views

connecting a raspberry pi to a fedora and a windows 7 machine

I am not sure where i would post this question, so please do not delete this question and just guide me to where i can post this question. My question is that by buying a switch would I be able to ...
0
votes
1answer
55 views

Distribute list of processes to many windows machines

I have a list of independent tests which run a common executable with different command lines and in different working directories. At the moment i simple run these one after the other in a batch ...
2
votes
1answer
65 views

Donate disk and CPU capacity to a third party? [closed]

Due to an impasse with one of our providers, we have come to have a non-trivial amount of CPU, and disk capacity that will stand unused for a number of months. How can I find services that will, ...
3
votes
1answer
1k views

Force most compilation to a remote host with distcc

The distcc FAQ states that there is currently no scheme for balancing or prioritizing hosts. They recommend to simply list the fastest host on top or the list. The situation here is that the remote ...
0
votes
1answer
67 views

The exact workflow of PVM (Parallel virtual machine)

These days I am learning PVM technology for concurrent processing. I go through so many blogs, tutorials to understand . Yes, I got some points, but no one gives me clear workflow of how it perform ...
4
votes
1answer
4k views

Distributed vs clustered vs networked file system

What is the differrence between a distributed file system, clustered file system and a network file system ?
0
votes
1answer
625 views

Free software/open source remote process execution and management system for linux?

I have the following requirements for a remote process execution/management system and thought I'd ask if anyone had seen something similar available under free software/open source licensing terms ...
1
vote
4answers
115 views

Distributed website server redundancy [closed]

Assume a website infrastructure is very complicated and is fully distributed (probably like most large web companies). Am I right in thinking that although there are all these extra web servers to ...
1
vote
2answers
452 views

Running BOINC on Debian

I run Debian unstable. I've installed boinc-client and boinc-manager packages. When i run boincmgr, it first says "Retrieving current status" and then "Unable to connect to the core client". http://...
1
vote
2answers
3k views

Open source Distributed computing tool

I want to set up distributed computing on my Local Area Network consisting a bunch of PCs. Say for the time being each one has the same OS - Windows 7. Is there any opensource tool available so that ...
2
votes
2answers
515 views

DFS for a several small clusters over WAN

My friends and I all have TBs on our system(s). None of us have any full backups which are geographically distributed however, because at that amount of data, solutions such as Dropbox, S3, et al. are ...
2
votes
1answer
531 views

How to put idle CPU processing power to good use? [closed]

I just got a new computer that has far more processing power than I need 90% of the time that I'm using it. To what ought I to devote that power? I know of SETI @ Home and Protein Folding. What ...
-1
votes
1answer
280 views

Distributed systems: how does their architecture allow them to do what is needed? [closed]

I was just wondering how these processes work, in an overarching sense, such as "distributed grep", "distributed compiling", "distributed sort", "distributed file systems", "distributed _______" Are ...
0
votes
1answer
505 views

What is a partitioned network? [closed]

I've learned what a partitioned hard disk is, but how does partitioning work when referring to distributed systems and computer networks?
3
votes
2answers
1k views

How do distributed filesystems differ from cluster filesystems?

How do distributed filesystems differ from cluster filesystems? Is it just the wording that I am getting tripped up on? Are there any significant differences between the two.
2
votes
1answer
352 views

What are “multikernel computers”?

Which does the term 'modern multikernel computers' refer to? Distributed systems or multi-core computers? I saw it in a one-line question, so no context to help! Google didn't help neither.
1
vote
1answer
1k views

Distributed File System using NFS?

Is it possible to setup NFS to perform as a parallel distributed file system? I'd like to have to NFS servers containing the same file data and serving file ops request for load balancing and to ...
3
votes
0answers
69 views

Manage processes on an distributed windows computer system

I want to manage the startup / re-start and shutdown of a multi module software system running on a set of windows machines. For example: [some machines booted, some still doing so] run the whole ...
1
vote
5answers
712 views

Is it possible to “farm” out graphics processing to a comparatively more powerful machine?

I have a computer with comparatively weak graphics, and one with comparatively strong graphics. I would like to run a graphics-intensive application on the weaker machine, using the graphics power of ...
0
votes
1answer
79 views

Are there any solutions where spare disk on workstations can be used as a distributed NAS?

With a windows based workstation heavy infrastructure, we have a lot of spare disk space. I am look about to see if there are any solutions where this space can be used as a sort of distributed NAS. ...
0
votes
1answer
380 views

SimpleBackUp with multiple hard drives

I have a lot of hard drives from old computers (6-7 years old). i want to get an external enclosure with USB 3.0 and use them for back up. To my understanding SimpleBackup copies everything the first ...
3
votes
2answers
2k views

Distributed Compiling on Windows

I read about distcc for distributed compiling on Linux systems. Since I have to compile a big project quite often, I was wondering if there are methods on Windows on how to distribute that onto ...
5
votes
4answers
1k views

Distributed, decentralized data sharing and backup

Consider the following scenario: you have a team of a few people scattered around the globe that needs to work on the same data. Each member of the team has a copy of the data he needs. When someone ...
1
vote
1answer
611 views

Run Folding@Home inside VMs on a single physical host, or just on the parent OS?

From a pure efficiency standpoint, would running 8 VMs on a 8 CPU core system, one in each of 8 1 CPU VMs be more efficient at processing the F@H data, or just running one instance on a dedicated ...
0
votes
1answer
482 views

Cloud Computing - Multiple Physical Computers, One Logical Computer

I know that you can set up multiple virtual machines per physical computer. I'm wondering if it's possible to make multiple physical computers behave as one logical unit? Fundamentally the way I ...
0
votes
2answers
728 views

torque in Cluster [closed]

I am learning and have some questions about using PBS to submit jobs in a cluster. (1) If a job is parallelized by MPI and run by mpirun without using qsub, it cab be running on different nodes. So ...
0
votes
3answers
1k views

Linux live-cd for distributed computing projects?

I have had this dream for years now... but sadly, even to this day, it remains just a dream. I want to find a live-cd that I can pop into any computer I have permission to use, and it boots up and ...
29
votes
9answers
27k views

Is there any way to distribute x264 encoding jobs across multiple computers (to increase the encoding speed)?

Does anyone know of a current, active solution to encoding x264 videos across many computers (via the network) to increase encoding FPS? Brownie points for cross-platform and open source, but just so ...