Last active
February 16, 2025 22:08
Revisions
-
jimmyFlash revised this gist
Feb 16, 2025 . 1 changed file with 75 additions and 5 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -3,6 +3,11 @@ - Networking command available on windows, Linux and macOS. - Curl stands for (Client URL) > [!hint] cURL on windows > Windows 10/11 come with curl built in them, you can use the curl command from PowerShell or cmd ### Basic Usage Basic usage will include the `curl` and a URL, this will basically return the html content of URL you input @@ -27,7 +32,7 @@ Date: Fri, 15 Mar 2024 22:48:52 GMT > [!info] using the `-i` flag will get you both the content + header > but that would be too much info to display - To output/download the content of the curl to a file, you can use the `-o` which comes built in `curl` and specify a file, or you can use the `-O` (capital O) flag **without a file name**, and that will use the name of the default html page served *in this case index.html* as the output. You can use `-s` to silence the status. ```shell > curl -o google.html https://www.google.com @@ -89,7 +94,7 @@ HTTPS: curl –insecure -i -X OPTIONS https://example.com/ ``` > You may also use `-v` (verbose) instead of `-i` to see more output to print both the request and response, or the `vvv` which gives more verbose info. > [!alert] `curl` against a site with self signed certificate @@ -100,7 +105,7 @@ curl –insecure -i -X OPTIONS https://example.com/ curl https://self-signed.badssl.com/ ``` You can use the `-k` to proceed to the site and skip certificate check. ```shell curl -k https://self-signed.badssl.com/ @@ -172,6 +177,13 @@ To send json data we need to us the flag `-H` to define the header "*content-typ curl -d '{"name":"test","salary":"123","age":"23"}' -H "content-type:application/json" https://dummy.restapiexample.com/api/v1/create ``` *Example 2.1*: using header shorthand `-H` with CSRF ```shell curl -X POST -H "X-CSRF: 1ndrgg9" -H "Origin: http://localhost:8080" -H "Content-Type: application/json" --data '{}' http://localhost:8080/csrf CSRF check passed! ``` *Example 3 : using the `--url` flag* ```shell @@ -254,12 +266,70 @@ MD5 F276B19AABCB4AE8CDA4D22625C6735F ``` ### Check public IP address ```cmd C:\Users\user>curl checkip.amazonaws.com ``` --- ### Fun stuff - **Get the weather** ```powershell C:\Users\user>curl wttr-in/Dubai ``` - **Un-shorted short links** ```cmd C:\Users\user>curl --head --location "https:/btl.short" | findstr location ``` - **Check website status** ```cmd C:\Users\user>curl -Is https://google.com ``` - Create a QR code ```cmd C:\Users\user>curl qrenco.de/{your website URL} ``` - Check latest video from favorite channel on YouTube, or X (Twitter) ```powershell %% Youtube %% C:\Users\user>curl -s https://decapi.me/youtube/latest_video?user=networkchuck %% X %% C:\Users\user>curl -s https://decapi.me/twitter/latest?user=networkchuck ``` - Check online dictionary for meaning of a word ```powershell C:\Users\user>curl dict.org/d:congress ``` --- ## Glossary | Options | | | --------------------- | -------------------------------------------------------------------------------- | | -V | # --version : show version number | | -h | # --help <category>: displays help menu or category if specified | | -o {file} | # --output: write to file | | -u user:pass | # --user: Authentication | | -v<br> | # --verbose | @@ -278,7 +348,7 @@ MD5 F276B19AABCB4AE8CDA4D22625C6735F | -G | # --get: send -d data via get | | **Headers** | | | -A {str} | # --user-agent | | -b 'name=val' | # --cookie | | -b FILE | # --cookie | | -H "X-Foo: y" | # --header | | --compressed | # use deflate/gzip | -
Aug 10, 2024 .There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -0,0 +1,284 @@ ## What is the Curl command ? - Networking command available on windows, Linux and macOS. - Curl stands for (Client URL) ### Basic Usage Basic usage will include the `curl` and a URL, this will basically return the html content of URL you input ```shell curl https://www.google.com ``` - To get just the header you'd use the `-I` flag ```shell > curl -I https://www.google.com HTTP/1.1 200 OK Content-Type: text/html; charset=ISO-8859-1 Content-Security-Policy-Report-Only: object-src 'none';base-uri 'self';script-src 'nonce-3nbjnYmAWAiUDuFeFgEZEA' 'strict-dynamic' 'report-sample' 'unsafe-eval' 'unsafe-inline' https: http:;report-uri https://csp.withgoogle.com/csp/gws/other-hp P3P: CP="This is not a P3P policy! See g.co/p3phelp for more info." Date: Fri, 15 Mar 2024 22:48:52 GMT ... ``` > [!info] using the `-i` flag will get you both the content + header > but that would be too much info to display - To output the content of the curl to a file, you can use the `-o` which comes built in `curl` and specify a file, or you can use the `-O` (capital O) flag **without a file name**, and that will use the name of the default html page served *in this case index.html* as the output ```shell > curl -o google.html https://www.google.com % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 20749 0 20749 0 0 6798 0 --:--:-- 0:00:03 --:--:-- 6805 # using the -O flag > curl -O https://www.google.com/index.html PS C:\Users\jamal\Desktop> curl -O https://www.google.com/index.html % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 20752 0 20752 0 0 13682 0 --:--:-- 0:00:01 --:--:-- 13706 ``` > [!important] If you don't specify a protocol like HTTPS for the URL > This could help you test for redirects to check if a site has HTTPS redirect - Testing redirect: ```shell > curl -I google.com HTTP/1.1 301 Moved Permanently Location: http://www.google.com/ Content-Type: text/html; charset=UTF-8 Content-Security-Policy-Report-Only: object-src 'none';base-uri 'self';script-src 'nonce-ac_yxVWLqCe8HOcnKSm7Yg' 'strict-dynamic' 'report-sample' 'unsafe-eval' 'unsafe-inline' https: http:;report-uri https://csp.withgoogle.com/csp/gws/other-hp Date: Fri, 15 Mar 2024 23:08:05 GMT Expires: Sun, 14 Apr 2024 23:08:05 GMT Cache-Control: public, max-age=2592000 Server: gws Content-Length: 219 X-XSS-Protection: 0 X-Frame-Options: SAMEORIGIN ``` - To follow redirects you add the `-L` flag. ```shell curl -IL google.com # adding the -v flag for verbose curl -ILv google.com ``` - You can test all methods for both http and https websites HTTP: ```shell curl -i -X OPTIONS http://example.com/ ``` HTTPS: ```shell curl –insecure -i -X OPTIONS https://example.com/ ``` > You may also use `-v` instead of `-i` to see more output. > [!alert] `curl` against a site with self signed certificate > if you attempt to use curl against domain with self signed certificate you get an error - below example this site uses a self signed certificate ```shell curl https://self-signed.badssl.com/ ``` you can use the `-k` to proceed to the site ```shell curl -k https://self-signed.badssl.com/ ``` --- ### Interacting with APIs You can use `curl` to test APIs. #### 1. fetching data with GET *Example 1: fetching list of employees* ```shell > curl https://dummy.restapiexample.com/api/v1/employees {"status":"success","data":[{"id":1,"employee_name":"Tiger Nixon","employee_salary":320800,"employee_age":61,"profile_image":""},{"id":2,"employee_name":"Garrett Winters","employee_salary":170750,"employee_age":63,"profile_image":""},{"id":3,"employee_name":"Ashton Cox","employee_salary":86000,"employee_age":66,"profile_image":""},{"id":4,"employee_name":"Cedric Kelly","employee_salary":433060,"employee_age":22,"profile_image":""},{"id":5,"employee_name":"Airi Satou","employee_salary":162700,"employee_age":33,"profile_image":""}, ... ],"message":"Successfully! All records has been fetched."} ``` *Example 2 : testing same endpoint with different query parameter* ( **this doesn't work in windows PowerShell)** ```shell > curl https://dummy.restapiexample.com/api/v1/employee/1 https://dummy.restapiexample.com/api/v1/employee/2 ``` *Example 3 : using regex with curl to search a range of parameters ( **this doesn't work in windows PowerShell**)* ```shell # fetching values for employee ids 1 to 5 curl https://dummy.restapiexample.com/api/v1/employee/[1-5] ``` *Example 4: basic authentication* ```shell curl -u user:pass -d status="Hello" http://twitter.com/statuses/update.xml ``` *Example 5 : file/data upload* ```shell # multipart file upload curl -v -include --form key1=value1 --form upload=<@localfilename> http://example.com/submit.cgi # multipart form: send data from text field and upload file curl -F person=anonymous -F secret=@file.txt http://example.com/submit.cgi ``` #### 2. Sending data with POST - The `-d` flag comes in handy, it's shorthand for **--data**, which is the body of the post request you want to send - When you add this flag the request is by defaulted to POST *Example 1: sending form data* ```shell curl -d name=jamal&salary=800&age=110 https://dummy.restapiexample.com/api/v1/create ``` > [!NOTE]- By default this method expects the payload to be form data types of key value pairs delimited by `&` *Example 2: sending json data* To send json data we need to us the flag `-H` to define the header "*content-type*" ```shell curl -d '{"name":"test","salary":"123","age":"23"}' -H "content-type:application/json" https://dummy.restapiexample.com/api/v1/create ``` *Example 3 : using the `--url` flag* ```shell curl --request POST --data "username=user1&password=test1234" -H 'Origin: https://securitylabs-ce.veracode.com' --url https://aeb7f888.community.ht/login ``` *Example 4 : sending bearer token in request header* ```shell curl --request GET --url https://aeb7f888.community.ht/users \ --header 'Origin: https://securitylabs-ce.veracode.com' \ --header 'X-Auth-Token: eyJhbGciOiAiSFMyNTYiLCAidHlwIjogIkpXVCJ9.eyJjbGFpbXMiOiB7ImxldmVsIjogInVzZXIiLCAidXNlcm5hbWUiOiAidXNlcjEifX0=.842611e2a755eaf54d0d05d8d84fab8cd6d35f3e4ea903bd743d605d7f2c9d87' ``` #### 3. Defining HTTP request methods Other than GET, POST what if we want to use other HTTP requests, you strictly specify the method you want to trigger using the `-X` flag *Example 1 : using DELETE method* ```shell curl -X DELETE https://dummy.restapiexample.com/api/v1/delete/2 ``` --- ### Host manipulation When your developing or debugging an app on your local machine and you want to simulate the host being a different domain. Use the flag `-- *Example 1: define a custom host header* ```shell curl --header "Host:example.com" http://127.0.0.1 ``` *Example 2 : custom resolve* let's resolve and address locally using the `--resolve` ```shell curl --resolve navek.org:443:127.0.0.1 https://navek.org/ ``` *Example 3: testing individual host , maybe if you are behind a load balancer and want to test a certain host, use the `--connect-to` flag to resolve to that host* ```shell curl --connect-to example.com:443:host-47.exmaple.com:443 https://example.com ``` --- ### Testing other protocols other than HTTP(s) With curl you can test ftp, LDAP,SMTP, sockets and Telnet *Example 1 : test Telnet* ```shell curl telnet:localhost:4317 ``` ### Advanced tricks Example1: Get the MD5 hash for the websites favicon image using `curl` + PowerShell ```powershell PS C:\Users\user> curl https://static-labs.tryhackme.cloud/sites/favicon/images/favicon.ico -UseBasicParsing -o favicon.ico Enter proxy password for user 'seBasicParsing': % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 1406 100 1406 0 0 811 0 0:00:01 0:00:01 --:--:-- 812 PS C:\Users\user> Get-FileHash .\favicon.ico -Algorithm MD5 Algorithm Hash Path --------- ---- ---- MD5 F276B19AABCB4AE8CDA4D22625C6735F C:\Users\user\favico… ``` --- ## Glossary | Options | | | --------------------- | -------------------------------------------------------------------------------- | | -o {file} | # --output: write to file | | -u user:pass | # --user: Authentication | | -v<br> | # --verbose | | -vv | # Even more verbose | | -s | # --silent: don't show progress meter or errors | | -S | # --show-error: when used with --silent (-sS), show errors but no progress meter | | -i | # --include: Include the HTTP-header in the output | | -I | # --head: headers only | | **Request** | | | -X {HTTP method} | # --request | | -L | # follow link if page redirects | | -F | # --form: HTTP POST data for multipart/form-data | | **Data** | | | -d<br>'data' | # --data: HTTP post data, URL encoded (eg, status="Hello") | | -d @file | # --data via file | | -G | # --get: send -d data via get | | **Headers** | | | -A {str} | # --user-agent | | -b name=val | # --cookie | | -b FILE | # --cookie | | -H "X-Foo: y" | # --header | | --compressed | # use deflate/gzip |