Trevor Sullivan
Trevor Sullivan•16mo ago

Getting Empty Output from Bash Script

I have a painfully simple Bash script that retrieves data from a publicly (anonymously) accessible REST API. Sometimes the script outputs the expected JSON, but the vast majority of the time, I'm not getting the JSON output from the curl command. What could possibly be happening? If I run this in a Linux shell, it works every single time.
54 Replies
Trevor Sullivan
Trevor Sullivan•16mo ago
The same command works perfectly fine in a Linux shell.
rubenf
rubenf•16mo ago
Let me investigate
Trevor Sullivan
Trevor Sullivan•16mo ago
I'm trying Python with httpx instead. Maybe it's more reliable that way. Was hoping to use Bash script for simplicity though
Trevor Sullivan
Trevor Sullivan•16mo ago
Python works perfectly, on the first try
rubenf
rubenf•16mo ago
bash relies on stdout for result while python works through file
Trevor Sullivan
Trevor Sullivan•16mo ago
Let me try using httpie from Bash script
rubenf
rubenf•16mo ago
it's just a race condition on stdout
Trevor Sullivan
Trevor Sullivan•16mo ago
sleep 1 after curl 😆
rubenf
rubenf•16mo ago
I can reproduce but this is really dumbfounding me it's not just a race condition
Trevor Sullivan
Trevor Sullivan•16mo ago
I found another odd behavior.
Trevor Sullivan
Trevor Sullivan•16mo ago
Trevor Sullivan
Trevor Sullivan•16mo ago
It works in Linux shell, but "not found" in Windmill
rubenf
rubenf•16mo ago
name is empty in your example
Trevor Sullivan
Trevor Sullivan•16mo ago
Oops
rubenf
rubenf•16mo ago
oh you mean pip3 install
Trevor Sullivan
Trevor Sullivan•16mo ago
Trevor Sullivan
Trevor Sullivan•16mo ago
Still the same But the Python example, with httpx works just fine
rubenf
rubenf•16mo ago
it's really weird
Trevor Sullivan
Trevor Sullivan•16mo ago
It's not a problem finding the http binary .... because I can do http --version and it prints the version
Trevor Sullivan
Trevor Sullivan•16mo ago
See, it prints the version, but not the output from the API call
rubenf
rubenf•16mo ago
if one pipe with jq it works everytime curl --silent https://api.nationalize.io?name=michael echo "" this will work everytime
Trevor Sullivan
Trevor Sullivan•16mo ago
Weird ...
rubenf
rubenf•16mo ago
it looks like the buffer is not getting drained but I think that's a bash thing
Trevor Sullivan
Trevor Sullivan•16mo ago
Yup, confirmed. That fixes the issue with curl. But with httpie I still just get "not found" Maybe separate issues
rubenf
rubenf•16mo ago
definitely separate
Trevor Sullivan
Trevor Sullivan•16mo ago
Seems like a variable expansion issue in Bash. I hard-coded a value and http works.
Trevor Sullivan
Trevor Sullivan•16mo ago
Sorry, that was coming from curl. Not HTTP. Hard-coding value still fails
rubenf
rubenf•16mo ago
does this command work on linux ?
Trevor Sullivan
Trevor Sullivan•16mo ago
Trevor Sullivan
Trevor Sullivan•16mo ago
Yup
rubenf
rubenf•16mo ago
i committed a fix that will avoid the need for echo ""
Trevor Sullivan
Trevor Sullivan•16mo ago
Do you want a new bug report for the "not found" issue?
rubenf
rubenf•16mo ago
I'm on it will probably solve it soon
rubenf
rubenf•16mo ago
rubenf
rubenf•16mo ago
I cannot reproduce
Trevor Sullivan
Trevor Sullivan•16mo ago
Hmm... Trying it another way, using a Docker container
rubenf
rubenf•16mo ago
note that i'm having the echo "" baked in as part of my new fix
Trevor Sullivan
Trevor Sullivan•16mo ago
Try this now ....
rubenf
rubenf•16mo ago
?
Trevor Sullivan
Trevor Sullivan•16mo ago
Calling API with PowerShell instead of http
Trevor Sullivan
Trevor Sullivan•16mo ago
Worked fine, first try
rubenf
rubenf•16mo ago
lol
Trevor Sullivan
Trevor Sullivan•16mo ago
Maybe something odd with the http utility ... really odd though. I'm not doing any HTTP proxying
rubenf
rubenf•16mo ago
Not found is really really weird can you do -v or something ?
Trevor Sullivan
Trevor Sullivan•16mo ago
Yeah, that's what I thought. I could expect many other errors, but not taht
Trevor Sullivan
Trevor Sullivan•16mo ago
Trevor Sullivan
Trevor Sullivan•16mo ago
http --verbose "https://api.nationalize.io?name=Trevor" That's .... bizarre? It's literally the same URL They must be doing some server-side check on the User-Agent header? But it works in Linux shell .....
rubenf
rubenf•16mo ago
user agent would be the same
Trevor Sullivan
Trevor Sullivan•16mo ago
Oh well, I have plenty of alternatives. I'll just use PowerShell or Python, or curl in Bash Yeah, just thought of that after I posted it Let me do --verbose from Linux shell
Trevor Sullivan
Trevor Sullivan•16mo ago
Trevor Sullivan
Trevor Sullivan•16mo ago
Older version of HTTPie Whatever, I'm not worried about it. At least you verified it's not an issue on your side.
rubenf
rubenf•16mo ago
GitHub
GitHub - PowerShell/PowerShell-Docker: Repository for building Powe...
Repository for building PowerShell Docker images. Contribute to PowerShell/PowerShell-Docker development by creating an account on GitHub.
rubenf
rubenf•16mo ago
220MB just for powershell...
Trevor Sullivan
Trevor Sullivan•16mo ago
Yup, but it saves tons of development time