Questions tagged [pipe]
Pipes or named pipes are a feature of the POSIX standard that allow separate processes to communicate with each other without having been designed explicitly to work together.
401
questions
1
vote
0
answers
17
views
CJK unicode characters output from `wine reg query` displayed on terminal are fine. But they become question marks after pipe
When I run wine reg query 'HKEY_CURRENT_USER\Software\Akelsoft\AkelPad\Recent' on my machine, the output is fine and contains something like file0 REG_MULTI_SZ Z:\home\x\怎.txt on my terminal.
...
- 175
2
votes
1
answer
121
views
Get `pv` work in `ssh` session
How do I get pv to work when called remotely via ssh?
E.g.
a@a-virtual-machine:~$ ssh [email protected] "pv < ~/Documents/Local/staging.sql.gz" | gunzip > /dev/null
This prints no ...
- 219
0
votes
1
answer
68
views
Grepping an specific command give me an old grep output
I need some help, wasn't able to find a solution...
I've made a script to run xfs_growfs and pipe its output to know only if the it found no difference is disk size and gave me an "data size ...
1
vote
1
answer
89
views
Piping commands to smtp server doesn't work
So I was trying to write a bash script to send a mail to someone using smtp.gmail.com server by using openssl.
Everything upto this is working fine:
printf "helo gmail.com\nauth plain $auth\n&...
- 11
0
votes
0
answers
581
views
How to take file name from yt-dlp and use it as ffmpeg output file name?
I'm trying to get the output name to be the same as the input name while using the pipe command in ffmpeg.
Example:
The file is called 1-Minute Audio Test for Stereo Speakers & Headphones [...
- 133
0
votes
0
answers
39
views
How to solve an audio stream encoding workflow thru pipes only
I have the following workflow which is typically executed on a set of provided audio files (can be MP3 files, as assumed in this example, but also different formats):
file000.mp3
file001.mp3
...
What ...
- 31
0
votes
0
answers
167
views
Is ffmpeg content decryption via named pipe input possible
I am trying to decrypt content that is cenc encoded via ffmpeg using the -cenc_decryption_key option
This works perfect if i point to the local nginx directory where my manifest.mpd file is located
...
0
votes
0
answers
140
views
Redirect pipe contains quotation marks to a variable in Windows batch file
I am trying to get the number of substrings "id" in a file and save to a file
I am trying to do it this way:
for /F %N in ('"C:\Program Files (x86)\GnuWin32\bin\grep.exe" -o "...
0
votes
0
answers
620
views
Redirect the Windows console output of a Python script without raising UnicodeEncodeError
I have a Python script called scratch_1.py which can be simplified for the sake of the question to:
s = "∞"
print(s)
If I run the script in the Windows console, I get the expected output:
...
- 187
2
votes
1
answer
195
views
Is there a way to Grep multiple HTML pages on a domain without downloading the files?
I essentially want to Ctrl+F multiple sites in a browser.
I have a range of URLs on a domain https://exampleblog.com/posts/{1-50}
I want to grep all of the pages under /posts/ at once. They are each ...
- 23
2
votes
0
answers
42
views
Cat with gun parallel does not continue as expected
Here is my code.
file_name="all_parts"
find parts/ -type f > ${file_name}
while read part
do
echo processing ${part}
cat ${part} | parallel -j 100 'wget {} -o logs/...
- 21
17
votes
3
answers
7k
views
Is pipe ( | ) a command?
As I understand it, pipe ( | ) takes the standard output of one process and passes it as standard input into another process.
But I want to know if pipe ( | ) is considered a command like ls, grep ...
- 321
0
votes
1
answer
346
views
Do shell pipes block upstream source processes if the buffer is overflowing?
I do various sysadmin tasks to clean up my disks, such as (but not limited to):
find /media/me/disk_with_huge_inode_count -type d -empty | xargs rmdir -p
and the rmdir part is really slow, while find ...
- 1,405
0
votes
1
answer
280
views
Bash split stream (tee) and join them together
I need to do something similar to this question, except that in that question the OP just concat the outputs of command2 and command3, and I need them be handed over separately, like this:
...
- 101
1
vote
1
answer
125
views
Piped commands altering standard output?
I've tried three different "alias" functions to make an image show from the command line given an argument. Can someone point me in the right direction towards figuring out why the input is ...
- 13
1
vote
1
answer
91
views
Build tool spawns a daemon that does not close its standard output. How can I prevent this from stalling my shell pipeline?
I am using a (closed-source) vendor-specific build tool (Microsemi Designer, an
FPGA layout tool). I invoke it from a shell script (highly simplified):
... # Setup
/opt/.../designer SCRIPT:my_script....
- 125
0
votes
0
answers
59
views
Pipes and file descriptors within bash scripts
Streaming PI camera video to VLC using the following command from bash prompt works OK:
raspivid -o - -t 0 -w 1280 -h 720 -fps 12 | cvlc stream:///dev/stdin --sout '#rtp{sdp=rtsp://:8081/}' :demux=...
- 251
0
votes
1
answer
2k
views
how pipe the 'get_url' of youtube-dl to ffmpeg stdin
I'm not getting pass the output from the first process with Youtube-dl through pipe to stdin of ffmpeg. It is as if the Youtube-dl force the execution of process after pipe instead wait for his own ...
- 1
0
votes
0
answers
519
views
Why can't I write to this pipe?
I followed the steps in the accepted answer in https://unix.stackexchange.com/questions/31824/how-do-i-attach-a-terminal-to-a-detached-process
However I get this
root@ca:/home/kali# echo blah > /...
- 101
1
vote
0
answers
169
views
Count lines passing through a windows shell pipe and stop after threshold is reached
I am on Windows, collecting a dataset and have to repeatedly perform following operations.
I'm looking for a way to automate some of them.
General description
Run a process that endlessly prints its ...
- 117
0
votes
1
answer
329
views
FFmpeg transcode piped mp4 with MOOV at end - but I can read it twice
I need to transcoding an MP4 file using that I can only access via a stream. The MOOV atom is at the end of the file. I do not have the ability to seek to a specific part of the file, only read a pipe ...
- 101
17
votes
4
answers
4k
views
What does "2>&1" do when posted BEFORE 1>x?
I know what this command does:
command 1>/dev/null 2>&1
But what, if anything, does the following do?
command 2>&1 1>/dev/null
I still see standard error output with the second ...
- 1,151
0
votes
0
answers
11
views
Why does tailing logs with kubectl prevent pipe from producing output? [duplicate]
I'm trying to process logs while tailing them.
This command works:
kubectl logs "$pod" --tail=100 | egrep pattern | cut -d ' ' -f 1
But this command produces no output:
kubectl logs "$...
- 958
0
votes
1
answer
515
views
What does " /bin/sh <$(tty) >$(tty) 2>$(tty) " mean?
I was looking at the GTFOBINS page for the at command and saw this command (last command in that page)
echo "/bin/sh <$(tty) >$(tty) 2>$(tty)" | sudo at now; tail -f /dev/null
for ...
- 47
0
votes
1
answer
224
views
Pipe Output of dhclient with sed
Goal
Limit output of dhclient.
Code
dhclient eth0|sed 's/answers//'
expected behaviour
RTNETLINK : Operation not permitted
got
RTNETLINK answers: Operation not permitted
Piping ifconfig works
ifconfig ...
- 148
0
votes
0
answers
144
views
Passing a piped variable to a sub-command called within GNU parallel
In bash, I am trying to find a list of files and pipe them to parallel.
find *.fasta | parallel "Gblocks {} -b2=$(( "$(cat {} | grep \> | wc -l)" / 2 + 1 ))"
Where Gblocks is a ...
0
votes
0
answers
738
views
Generate GPG key without saving to keychain
I am trying to generate a GPG key with gnupg. However, instead of exporting the secret key to a file, I am trying to pipe it to another program (keybase pgp import) that will process it.
Right now, I ...
- 141
1
vote
1
answer
251
views
Git for Windows leaking processes under WSL
I work in Debian under WSL2, but because my project files live in the Windows filesystem, I've been using Git for Windows through the WSL interop with a shell function like this:
git() {
case &...
- 113
0
votes
1
answer
2k
views
Posting file to pastebin via Batch with CURL
I am attempting to upload log files to pastebin using CURL.exe on a Windows 10 machine in a batch (.bat) file.
The following command works:
curl -X POST -d "api_dev_key=MY_DEV_KEY" -d "...
- 1,508
0
votes
1
answer
3k
views
ffmpeg '-i pipe:' and 'thread_queue_size' -- managing a pipe
I've discovered a thread_queue_size number that seems to work on my system for this use-case but I'll test more (or repeat tests) to help the cause if someone can suggest something. Please advise. ...
- 11
1
vote
0
answers
428
views
FFmpeg concat of mov files via pipe input fails on versions newer than 4.2.3
I've been using ffmpeg 4.2.3 to concat h264 mov files from a DSLR camera using the following:
ls input1.MOV input2.MOV | perl -ne 'print "file $_"' | ffmpeg -f concat -protocol_whitelist ...
- 11
0
votes
1
answer
702
views
zsh fails to interpret a piped command which is OK with bash
~ ᐅ docker image inspect nginx | jq .[].Config.ExposedPorts
zsh: no matches found: .[].Config.ExposedPorts
With bash it works as expected.
I found out a difference as to how the piping is interpreted ...
- 111
0
votes
0
answers
2k
views
Use named pipes in ffmpeg
In my application I want to modify various mp3 and then mix them together. I know I could do it with a single command line in FFmpeg but it can end up very messy since I need to use various filter on ...
- 101
1
vote
1
answer
148
views
How does a Unix program know to change output if it's being piped?
Calling something like ag (a grep alternative) results in an output that's human readable. Results are printed with headers. Useful for humans, horrible for programs.
$ ag 'filter\('
tests/...
- 1,287
0
votes
1
answer
655
views
How to convert binary input into utf-16 characters on bash
I have bunch of bytes that has random bits in order and some of them 9 and 10. I can convert the 8 bit binary data into characters with a perl script like below.
perl -lape '$_=pack"(B8)*",@...
- 15
0
votes
1
answer
398
views
Script runs slow processing stdout piped from service
I am using a utility named oscdump which listens on a specified port for OSC messages and prints the messages to stdout. I wish to parse and process these messages in a script however my script runs ...
- 63
1
vote
0
answers
999
views
Send ffmpeg segments to stdout so they can be processed by next pipe
I have a pipeline like
ffmpeg -i input.mov -f matroska - |\
ffmpeg -i - -codec copy -map 0 -reset_timestamps 1 \
-segment_time 10 -f segment output/out%03d.mov
which works and it is splitting the ...
- 111
1
vote
1
answer
148
views
Delete trailing hashes in a list of URLs using powershell?
I got a bit rusty with PowerShell, and I am missing the correct search terms, so I hoping asking here is ok.
I have list of URL of the following structure:
/pub/pdf/ssm.supervisorymanual.en.pdf?...
- 204
4
votes
2
answers
4k
views
ffmpeg less verbose, without hiding progress bar
Without editing and recompiling ffmpeg from source, how can one hide some of the many lines that it prints when it starts encoding, without also hiding its progress bar that updates every second or so ...
- 2,299
0
votes
0
answers
829
views
Why does a linux pipe slow my data transfer?
I noticed that if I use dd to move data from /dev/zero to /dev/null, I can max out around 6.1GB/s by adjusting the block size.
dd if=/dev/zero of=/dev/null status=progress bs=512k count=2k
# 2048+0 ...
- 101
0
votes
1
answer
301
views
How can the convert command's output be passed/piped instead of saved to a file?
I made a HTML file with images as base64. I started bash-scripting to make a folders images be base64-printed into this html-file. I worked, I share the code:
cat html_head.txt > fil.html
for i in *...
0
votes
1
answer
987
views
How to netcat to multiple ports
I'am using a raspberry and i need 2 local streams. This is what i've tried:
Attempt
raspivid <some options> -o - | tee nc localhost 5100 | nc localhost 5000
Question
I can receive the output on ...
- 103
1
vote
1
answer
103
views
windows 10: process 1 output used as process 2 input: pipes
process 1 :
adb shell screenrecord --bit-rate 6000000 --size 1280x720 --output-format=h264 -
process 2:
ffplay -f h264
normally ffplay takes its input as an argument (-i _input)
How to pipe those ...
- 113
1
vote
1
answer
388
views
Piping Unrar Output to FFMPEG
I have a password protected .rar file. It's large, so using unrar e large.rar is time consuming. Once extracted, the file will be large.avi
Then I have to do ffmpeg -i large.avi large.mp4
I was hoping ...
0
votes
0
answers
91
views
Circular pipe in bash
I need a circular pipe in Bash.
This doesn’t work:
exec 3>&1 | socat | awk | socat >&3 #exec 3>&1
This works:
exec 3> /dev/stderr ; echo test >&3
Can somebody provide ...
- 1
1
vote
1
answer
194
views
"Upload" pipe as if it were a regular file
Suppose I want to tar and compress a file for upload to Google Drive or something, but all without ever saving an intermediate tarball (e.g. because of a lack of disk space and/or the disk is read-...
- 387
0
votes
0
answers
149
views
When piping sed command, output becomes all spaces
This is for a lab I have in a class, the professor has told me to use sed to remove all punctuation and double spacing by piping together several sed commands. When I pipe the commands together my ...
1
vote
0
answers
78
views
How to launch a subprocess with captured output, but letting the subprocess believe it isn't captured?
Imagine launching a subprocess say my_binary in these two ways (I'm using bash for simplicity, but I assume this applies for other languages on POSIX systems):
#!/bin/bash
# 1) inherit stdout/stderr
...
- 225
3
votes
1
answer
328
views
Obtain chunk size in split --filter commands?
I'd like to compress a partclone backup on the fly using zstd. Instead of writing one large zstd Frame, the archive should be split into small chunks that allow semi-random access. The latter requires ...
- 141
1
vote
2
answers
2k
views
Unattended WSL Ubuntu 18.04 installation from powershell
Scenario
To automatically install and initialize WSL Ubuntu 18.04 with powershell, I am trying to initialize/set the first username and password automatically. However, when I first run the wsl from a ...
- 383