https://github.com/mikefarah/yq

Skip to content Toggle navigation
 
Sign up

  * Product
      +  
        Actions
        Automate any workflow
      +  
        Packages
        Host and manage packages
      +  
        Security
        Find and fix vulnerabilities
      +  
        Codespaces
        Instant dev environments
      +  
        Copilot
        Write better code with AI
      +  
        Code review
        Manage code changes
      +  
        Issues
        Plan and track work
      +  
        Discussions
        Collaborate outside of code
      + Explore
      + All features
      + Documentation
      + GitHub Skills
      + Blog
  * Solutions
      + For
      + Enterprise
      + Teams
      + Startups
      + Education
      + By Solution
      + CI/CD & Automation
      + DevOps
      + DevSecOps
      + Case Studies
      + Customer Stories
      + Resources
  * Open Source
      +  
        GitHub Sponsors
        Fund open source developers
      +  
        The ReadME Project
        GitHub community articles
      + Repositories
      + Topics
      + Trending
      + Collections
  * Pricing

[                    ] 

  *  
    #
    In this repository All GitHub |
    Jump to |

  * No suggested jump to results

  *  
    #
    In this repository All GitHub |
    Jump to |
  *  
    #
    In this user All GitHub |
    Jump to |
  *  
    #
    In this repository All GitHub |
    Jump to |

Sign in
Sign up
{{ message }}
mikefarah / yq Public

  * 
  * Notifications
  * Fork 420
  * Star 7.5k

yq is a portable command-line YAML, JSON, XML, CSV and properties
processor

mikefarah.gitbook.io/yq/

License

MIT license
7.5k stars 420 forks
Star
Notifications

  * Code
  * Issues 79
  * Pull requests 3
  * Discussions
  * Actions
  * Projects 0
  * Security
  * Insights

More

  * Code
  * Issues
  * Pull requests
  * Discussions
  * Actions
  * Projects
  * Security
  * Insights

mikefarah/yq

This commit does not belong to any branch on this repository, and may
belong to a fork outside of the repository.

Use this GitHub Action with your project

Add this Action to an existing workflow or create a new one.

View on Marketplace
master
Switch branches/tags
[                    ]
Branches Tags
Could not load branches
Nothing to show
{{ refName }} default View all branches
Could not load tags
Nothing to show
{{ refName }} default
View all tags

Name already in use

A tag already exists with the provided branch name. Many Git commands
accept both tag and branch names, so creating this branch may cause
unexpected behavior. Are you sure you want to create this branch?
Cancel Create
59 branches 155 tags
Code

  * Local
  * Codespaces

  *  
    Clone
    HTTPS GitHub CLI
    [https://github.com/m]

    Use Git or checkout with SVN using the web URL.

    [gh repo clone mikefa]

    Work fast with our official CLI. Learn more.

  * Open with GitHub Desktop
  * Download ZIP

Sign In Required

Please sign in to use Codespaces.

Launching GitHub Desktop

If nothing happens, download GitHub Desktop and try again.

Launching GitHub Desktop

If nothing happens, download GitHub Desktop and try again.

Launching Xcode

If nothing happens, download Xcode and try again.

Launching Visual Studio Code

Your codespace will open once ready.

There was a problem preparing your codespace, please try again.

Latest commit

@mikefarah
mikefarah Fixed date comparison with string date #1537
...
88a6b20 Feb 2, 2023
Fixed date comparison with string date #1537
88a6b20

Git stats

  * 1,494 commits

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
.github
Fixing versioning code for release
November 15, 2022 20:08
acceptance_tests
Include blank new lines in leading header preprocessing #1462
December 8, 2022 13:33
cmd
Bumping version
January 15, 2023 11:37
debian
Update changelog for version v4.16.2 (#1046)
January 3, 2022 09:20
examples
XML Decoder: Checking for invalid content outside of a root node #
1448
November 27, 2022 17:46
github-action
Fix broken multiline strings from dee0c94 (#1390)
October 19, 2022 10:16
pkg/yqlib
Fixed date comparison with string date #1537
February 2, 2023 13:30
scripts
Fixing versioning code for release
November 15, 2022 20:08
snap
Bumping version
January 15, 2023 11:37
test
Fixed load operator bug
October 25, 2022 13:30
.dockerignore
Added gosec
March 3, 2021 19:44
.gitignore
Bumped go compiler; bumped quality tools
August 29, 2022 14:13
.golangci.yml
refactor: move from io/ioutil to io and os packages (#1068)
January 21, 2022 19:50
CODE_OF_CONDUCT.md
Create CODE_OF_CONDUCT.md
February 21, 2020 21:13
CONTRIBUTING.md
Added contributing guide on documentation
July 7, 2022 11:53
Dockerfile
Bump golang from 1.19.4 to 1.19.5 (#1516)
January 12, 2023 14:21
Dockerfile.dev
Bump golang from 1.19.4 to 1.19.5 (#1516)
January 12, 2023 14:21
LICENSE
Added XML decoder (#1044)
December 21, 2021 15:02
Makefile
Add support for Podman as well as Docker (#1026)
December 2, 2021 10:01
Makefile.variables
Add support for Podman as well as Docker (#1026)
December 2, 2021 10:01
README.md
Fixed doc regarding getting the latest version
November 8, 2022 08:50
action.yml
action uses built docker image
October 19, 2022 11:06
go.mod
Upgrading golang.org/x/net package (#1540)
February 2, 2023 11:56
go.sum
Upgrading golang.org/x/net package (#1540)
February 2, 2023 11:56
how-it-works.md
chore: remove duplicate word in doc (#1322)
August 31, 2022 18:43
mkdocs.yml
Removed user docs
January 30, 2020 10:32
release_instructions.txt
updating release notes
November 28, 2022 08:39
release_notes.txt
Log info message when unable to chown file #1521
January 15, 2023 11:36
utf8.csv
Fixed handling of UTF8 encoded CSVs #1373
October 8, 2022 13:12
yq.go
Fix zsh completion (#1108)
February 14, 2022 08:03
View code
[                    ]
yq Quick Usage Guide Install Download the latest binary wget
Compressed via tar.gz Plain binary Latest version MacOS / Linux via
Homebrew: Linux via snap: Snap notes Run with Docker or Podman
Oneshot use: Pipe in via STDIN: Run commands interactively: Running
as root: Missing timezone data Podman with SELinux GitHub Action Go
Install: Community Supported Installation methods Nix Webi Arch Linux
Windows: Winget Mac: Alpine Linux On Ubuntu 16.04 or higher from
Debian package: Features Usage Known Issues / Missing Features

README.md

 yq

Build Docker Pulls Github Releases (by Release) Go Report CodeQL

a lightweight and portable command-line YAML, JSON and XML processor.
yq uses jq like syntax but works with yaml files as well as json,
xml, properties, csv and tsv. It doesn't yet support everything jq
does - but it does support the most common operations and functions,
and more is being added continuously.

yq is written in go - so you can download a dependency free binary
for your platform and you are good to go! If you prefer there are a
variety of package managers that can be used as well as Docker and
Podman, all listed below.

 Quick Usage Guide

Read a value:

yq '.a.b[0].c' file.yaml

Pipe from STDIN:

yq '.a.b[0].c' < file.yaml

Update a yaml file, inplace

yq -i '.a.b[0].c = "cool"' file.yaml

Update using environment variables

NAME=mike yq -i '.a.b[0].c = strenv(NAME)' file.yaml

Merge multiple files

# note the use of `ea` to evaluate all the files at once
# instead of in sequence
yq ea '. as $item ireduce ({}; . * $item )' path/to/*.yml

Multiple updates to a yaml file

yq -i '
  .a.b[0].c = "cool" |
  .x.y.z = "foobar" |
  .person.name = strenv(NAME)
' file.yaml

Convert JSON to YAML

yq -P sample.json

See the documentation for more examples.

Take a look at the discussions for common questions, and cool ideas

 Install

 Download the latest binary

 wget

Use wget to download, gzipped pre-compiled binaries:

For instance, VERSION=v4.2.0 and BINARY=yq_linux_amd64

 Compressed via tar.gz

wget https://github.com/mikefarah/yq/releases/download/${VERSION}/${BINARY}.tar.gz -O - |\
  tar xz && mv ${BINARY} /usr/bin/yq

 Plain binary

wget https://github.com/mikefarah/yq/releases/download/${VERSION}/${BINARY} -O /usr/bin/yq &&\
    chmod +x /usr/bin/yq

 Latest version

wget https://github.com/mikefarah/yq/releases/latest/download/yq_linux_amd64 -O /usr/bin/yq &&\
    chmod +x /usr/bin/yq

 MacOS / Linux via Homebrew:

Using Homebrew

brew install yq

 Linux via snap:

snap install yq

 Snap notes

yq installs with strict confinement in snap, this means it doesn't
have direct access to root files. To read root files you can:

sudo cat /etc/myfile | yq '.a.path'

And to write to a root file you can either use sponge:

sudo cat /etc/myfile | yq '.a.path = "value"' | sudo sponge /etc/myfile

or write to a temporary file:

sudo cat /etc/myfile | yq '.a.path = "value"' | sudo tee /etc/myfile.tmp
sudo mv /etc/myfile.tmp /etc/myfile
rm /etc/myfile.tmp

 Run with Docker or Podman

 Oneshot use:

docker run --rm -v "${PWD}":/workdir mikefarah/yq [command] [flags] [expression ]FILE...

Note that you can run yq in docker without network access and other
privileges if you desire, namely --security-opt=no-new-privileges
--cap-drop all --network none.

podman run --rm -v "${PWD}":/workdir mikefarah/yq [command] [flags] [expression ]FILE...

 Pipe in via STDIN:

You'll need to pass the -i\--interactive flag to docker:

docker run -i --rm mikefarah/yq '.this.thing' < myfile.yml

podman run -i --rm mikefarah/yq '.this.thing' < myfile.yml

 Run commands interactively:

docker run --rm -it -v "${PWD}":/workdir --entrypoint sh mikefarah/yq

podman run --rm -it -v "${PWD}":/workdir --entrypoint sh mikefarah/yq

It can be useful to have a bash function to avoid typing the whole
docker command:

yq() {
  docker run --rm -i -v "${PWD}":/workdir mikefarah/yq "$@"
}

yq() {
  podman run --rm -i -v "${PWD}":/workdir mikefarah/yq "$@"
}

 Running as root:

yq's container image no longer runs under root (#860). If you'd like
to install more things in the container image, or you're having
permissions issues when attempting to read/write files you'll need to
either:

docker run --user="root" -it --entrypoint sh mikefarah/yq

podman run --user="root" -it --entrypoint sh mikefarah/yq

Or, in your Dockerfile:

FROM mikefarah/yq

USER root
RUN apk add --no-cache bash
USER yq

 Missing timezone data

By default, the alpine image yq uses does not include timezone data.
If you'd like to use the tz operator, you'll need to include this
data:

FROM mikefarah/yq

USER root
RUN apk add --no-cache tzdata
USER yq

 Podman with SELinux

If you are using podman with SELinux, you will need to set the shared
volume flag :z on the volume mount:

-v "${PWD}":/workdir:z

 GitHub Action

  - name: Set foobar to cool
    uses: mikefarah/yq@master
    with:
      cmd: yq -i '.foo.bar = "cool"' 'config.yml'
  - name: Get an entry with a variable that might contain dots or spaces
    id: get_username
    uses: mikefarah/yq@master
    with:
      cmd: yq '.all.children.["${{ matrix.ip_address }}"].username' ops/inventories/production.yml
  - name: Reuse a variable obtained in another step
    run: echo ${{ steps.get_username.outputs.result }}

See https://mikefarah.gitbook.io/yq/usage/github-action for more.

 Go Install:

go install github.com/mikefarah/yq/v4@latest

 Community Supported Installation methods

As these are supported by the community [?] - however, they may be out
of date with the officially supported releases.

 Nix

nix profile install nixpkgs#yq-go

See here

 Webi

webi yq

See webi Supported by @adithyasunil26 (https://github.com/webinstall/
webi-installers/tree/master/yq)

 Arch Linux

pacman -S go-yq

 Windows:

Chocolatey Chocolatey

choco install yq

Supported by @chillum (https://chocolatey.org/packages/yq)

and

 Winget

winget install yq

https://winget.run/pkg/MikeFarah/yq

 Mac:

Using MacPorts

sudo port selfupdate
sudo port install yq

Supported by @herbygillot (https://ports.macports.org/maintainer/
github/herbygillot)

 Alpine Linux

  * Enable edge/community repo by adding $MIRROR/alpine/edge/
    community to /etc/apk/repositories
  * Update database index with apk update
  * Install yq with apk add yq

Supported by Tuan Hoang https://pkgs.alpinelinux.org/package/edge/
community/x86/yq

 On Ubuntu 16.04 or higher from Debian package:

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys CC86BB64
sudo add-apt-repository ppa:rmescandon/yq
sudo apt update
sudo apt install yq -y

Supported by @rmescandon (https://launchpad.net/~rmescandon/+archive/
ubuntu/yq)

 Features

  * Detailed documentation with many examples
  * Written in portable go, so you can download a lovely dependency
    free binary
  * Uses similar syntax as jq but works with YAML, JSON and XML files
  * Fully supports multi document yaml files
  * Supports yaml front matter blocks (e.g. jekyll/assemble)
  * Colorized yaml output
  * Date/Time manipulation and formatting with TZ
  * Deeply data structures
  * Sort keys
  * Manipulate yaml comments, styling, tags and anchors and aliases.
  * Update inplace
  * Complex expressions to select and update
  * Keeps yaml formatting and comments when updating (though there
    are issues with whitespace)
  * Decode/Encode base64 data
  * Load content from other files
  * Convert to/from json/ndjson
  * Convert to/from xml
  * Convert to/from properties
  * Convert to/from csv/tsv
  * General shell completion scripts (bash/zsh/fish/powershell)
  * Reduce to merge multiple files or sum an array or other fancy
    things.
  * Github Action to use in your automated pipeline (thanks
    @devorbitus)

 Usage

Check out the documentation for more detailed and advanced usage.

Usage:
  yq [flags]
  yq [command]

Examples:

# yq defaults to 'eval' command if no command is specified. See "yq eval --help" for more examples.
yq '.stuff' < myfile.yml # outputs the data at the "stuff" node from "myfile.yml"

yq -i '.stuff = "foo"' myfile.yml # update myfile.yml inplace


Available Commands:
  completion       Generate the autocompletion script for the specified shell
  eval             (default) Apply the expression to each document in each yaml file in sequence
  eval-all         Loads _all_ yaml documents of _all_ yaml files and runs expression once
  help             Help about any command
  shell-completion Generate completion script

Flags:
  -C, --colors                        force print with colors
  -e, --exit-status                   set exit status if there are no matches or null or false is returned
  -f, --front-matter string           (extract|process) first input as yaml front-matter. Extract will pull out the yaml content, process will run the expression against the yaml content, leaving the remaining data intact
      --header-preprocess             Slurp any header comments and separators before processing expression. (default true)
  -h, --help                          help for yq
  -I, --indent int                    sets indent level for output (default 2)
  -i, --inplace                       update the file inplace of first file given.
  -p, --input-format string           [yaml|y|xml|x] parse format for input. Note that json is a subset of yaml. (default "yaml")
  -M, --no-colors                     force print with no colors
  -N, --no-doc                        Don't print document separators (---)
  -n, --null-input                    Don't read input, simply evaluate the expression given. Useful for creating docs from scratch.
  -o, --output-format string          [yaml|y|json|j|props|p|xml|x] output format type. (default "yaml")
  -P, --prettyPrint                   pretty print, shorthand for '... style = ""'
  -s, --split-exp string              print each result (or doc) into a file named (exp). [exp] argument must return a string. You can use $index in the expression as the result counter.
      --unwrapScalar                  unwrap scalar, print the value with no quotes, colors or comments (default true)
  -v, --verbose                       verbose mode
  -V, --version                       Print version information and quit
      --xml-attribute-prefix string   prefix for xml attributes (default "+")
      --xml-content-name string       name for xml content (if no attribute name is present). (default "+content")

Use "yq [command] --help" for more information about a command.

 Known Issues / Missing Features

  * yq attempts to preserve comment positions and whitespace as much
    as possible, but it does not handle all scenarios (see https://
    github.com/go-yaml/yaml/tree/v3 for details)
  * Powershell has its own...opinions on quoting yq

See tips and tricks for more common problems and solutions.

About

yq is a portable command-line YAML, JSON, XML, CSV and properties
processor

mikefarah.gitbook.io/yq/

Topics

bash cli golang yaml json csv portable xml properties devops-tools 
splat yaml-processor

Resources

Readme

License

MIT license

Code of conduct

Code of conduct

Stars

7.5k stars

Watchers

51 watching

Forks

420 forks

Releases 124

 
v4.30.8 - Snap fix Latest
Jan 15, 2023
+ 123 releases

Sponsor this project

 
Sponsor
Learn more about GitHub Sponsors

Used by 42

 

  * @lawliet89
  * @judedaryl
  * @garybowers
  * @L3n41c
  * @charithmadhuranga
  * @QuangHuy2705
  * @loblaw-sre

+ 34

Contributors 82

  * @mikefarah
  * @dependabot[bot]
  * @kenjones-cisco
  * @rmescandon
  * @mmorel-35
  * @inspirit1986
  * @cjnosal
  * @mkatychev
  * @xpzouying
  * @RyanSiu1995
  * @thehowl

+ 71 contributors

Languages

  * Go 87.1%
  * Shell 12.4%
  * Other 0.5%

Footer

 (c) 2023 GitHub, Inc.

Footer navigation

  * Terms
  * Privacy
  * Security
  * Status
  * Docs
  * Contact GitHub
  * Pricing
  * API
  * Training
  * Blog
  * About

You can't perform that action at this time.
You signed in with another tab or window. Reload to refresh your
session. You signed out in another tab or window. Reload to refresh
your session.