pub/scm/linux/kernel/git/sj/damo.git  about / heads / tags
DAMON user-space tool. Stands for Data Access Monitoring Operator
$ git log --pretty=format:'%h %s (%cs)%d'
35ef0f8 Update the version (2024-05-20)
	(HEAD -> master, tag: v2.3.5, next)
3cc84cc src/damo_show: De-duplicate sorted_regions()' dsc calculation (2024-05-19)
439744e src/damo_show: Add 'temperature' sorting key (2024-05-19)
3f57205 release_note: Update for next release (2024-05-19)
21e3de1 TODO: Update (2024-05-19)
30f38bc src/_damon_args: Implement --damos_quota_space option (2024-05-19)
f53196e src/_damon_args: Impelement --damos_quota_time option (2024-05-19)
0987947 src/_damon_args: Add --damos_quota_weight option (2024-05-19)
48d6781 src/_damon_args: Add --damos_quota_interval option (2024-05-19)
5266439 src/_damon_args: Split quota building logic (2024-05-19)
...

$ git cat-file blob HEAD:README.md
DAMO: Data Access Monitoring Operator
=====================================

`damo` is a user space tool for [DAMON](https://damonitor.github.io).  Using
this, you can monitor the data access patterns of your system or workloads and
make data access-aware memory management optimizations.

![damo monitor demo for water_nsquared](images/damo_monitor_water_nsquared.gif)


Demo Video
==========

Please click the below thumbnail to show the short demo video.

[![DAMON: a demo for the Kernel Summit 2020](
http://img.youtube.com/vi/l63eqbVBZRY/0.jpg)](
http://www.youtube.com/watch?v=l63eqbVBZRY
"DAMON: a demo for the Kernel Summit 2020")


Getting Started
===============

[![Packaging status](https://repology.org/badge/vertical-allrepos/damo.svg)](https://repology.org/project/damo/versions)

Follow below instructions and commands to monitor and visualize the access
pattern of your workload.

    $ # ensure DAMON is enabled on your kernel
    $ # install damo using packaging systems listed above,
    $ # or cloning the source repo and updating $PATH.
    $ sudo damo show $(pidof <your workload>)
    $ sudo damo record $(pidof <your workload>)
    $ sudo damo report heats --heatmap stdout --stdout_heatmap_color emotion

The second and last commands will show the access pattern of your workload,
like below:

![masim_stairs_snapshot](images/masim_stairs_snapshot.png)
![masim stairs heatmap in ascii](images/masim_stairs_heatmap_ascii.png)


FAQs
====

How can I ensure DAMON is enabled on my kernel?
-----------------------------------------------

Please refer to 'Install'
[section](https://sjp38.github.io/post/damon/#install) of the project webpage.

Where can I get more detailed usage?
------------------------------------

The below sections provide quick introductions for `damo`'s major features.
For more detailed usage, please refer to [USAGE.md](USAGE.md) file.


What does the version numbers mean?
-----------------------------------

Nothing at all but indicate which version is more fresh.  A higher version
number means it is more recently released.


Will `pip3 install damo` install the latest version of `damo`?
--------------------------------------------------------------

It will install the latest _stable_ version of `damo`.  If you want, you can
also install less stable but more fresh `damo` from source code.  For that,
fetch the `next` branch of the source tree and use `damo` executable file in
the tree.

    $ git clone https://github.com/awslabs/damo -b next


How can I participate in the development of `damo`?
---------------------------------------------------

Please refer to
[CONTRIBUTING](https://github.com/awslabs/damo/blob/next/CONTRIBUTING) file.


Why some features are not documented on [USAGE.md](USAGE.md) file?
------------------------------------------------------------------

Because those are not yet stabilized.  In other words, such features are in
their experimental stages, and therefore could be deprecated and removed
without notice and grace periods.  The documented features could also be
deprecated, but those will provide some notifications and grace periods.  If
there are some features you're relying on but not documented, please
[report](REPORTING.md) your usecase to the community.


How can I know deprecation schedule and alternatives of specific features?
--------------------------------------------------------------------------

The list of features that will be, or already deprecated, the deprecation
schedule, and their alternatives are summarized in
[FEATURES_DEPRECATION_SCHEDULE.md](FEATURES_DEPRECATION_SCHEDULE.md) file.


Quick Intro for Major Features
==============================

Below are quick introductions for `damo`'s major features.
For more detailed usage, please refer to [USAGE.md](USAGE.md) file.


Snapshot Data Access Pattern
----------------------------

Below commands repeatedly get a snapshot of the access pattern of a program for
every second.

    $ git clone https://github.com/sjp38/masim
    $ cd masim; make; ./masim ./configs/zigzag.cfg --silent &
    $ sudo damo start --target_pid $(pidof masim)
    $ while :; do sudo damo show; sleep 1; done

The first two lines of the commands get an artificial memory access generator
program and run it in the background.  It will repeatedly access two 100
MiB-sized memory regions one by one.  You can substitute this with your real
workload.

The third line asks ``damo`` to start monitoring the access pattern of the
process.  Finally, the last line retries a snapshot of the monitoring results
every second and show on terminal.


Recording Data Access Patterns
------------------------------

Below commands record memory access patterns of a program and save the
monitoring results in `damon.data` file.

    $ git clone https://github.com/sjp38/masim
    $ cd masim; make; ./masim ./configs/zigzag.cfg --silent &
    $ sudo damo record -o damon.data $(pidof masim)

The first two lines of the commands get an artificial memory access generator
program and run it in the background.  It will repeatedly access two 100
MiB-sized memory regions one by one.  You can substitute this with your real
workload.  The last line asks ``damo`` to record the access pattern in
``damon.data`` file.


Visualizing Recorded Patterns
-----------------------------

Below three commands visualize the recorded access patterns into three
image files.

    $ damo report heats --heatmap stdout
    $ damo report wss --range 0 101 1
    $ damo report wss --range 0 101 1 --sortby time --plot

- ``access_pattern_heatmap.png`` will show the data access pattern in a
  heatmap, which shows when (x-axis) what memory region (y-axis) is how
  frequently accessed (color).
- ``wss_dist.png`` will show the distribution of the working set size.
- ``wss_chron_change.png`` will show how the working set size has
  chronologically changed.

You can show the images on a web page [1].  Those made with other realistic
workloads are also available [2,3,4].

[1] https://damonitor.github.io/doc/html/latest/admin-guide/mm/damon/start.html#visualizing-recorded-patterns<br>
[2] https://damonitor.github.io/test/result/visual/latest/rec.heatmap.1.png.html<br>
[3] https://damonitor.github.io/test/result/visual/latest/rec.wss_sz.png.html<br>
[4] https://damonitor.github.io/test/result/visual/latest/rec.wss_time.png.html


Data Access Pattern Aware Memory Management
-------------------------------------------

Below command makes every memory region of size >=4K that hasn't accessed for
>=60 seconds in your workload to be swapped out.  By doing this, you can make
your workload more memory efficient with only modest performance overhead.

    $ sudo damo start --damos_access_rate 0 0 --damos_sz_region 4K max \
                        --damos_age 60s max --damos_action pageout \
                        <pid of your workload>

# heads (aka `branches'):
$ git for-each-ref --sort=-creatordate refs/heads \
	--format='%(HEAD) %(refname:short) %(subject) (%(creatordate:short))'
* master       Update the version (2024-05-20)
  next         Update the version (2024-05-20)

# tags:
$ git for-each-ref --sort=-creatordate refs/tags \
	--format='%(refname:short) %(subject) (%(creatordate:short))'
v2.3.5       DAMO 2.3.5 (2024-05-20) tar.gz
v2.3.4       DAMO 2.3.4 (2024-05-12) tar.gz
v2.3.3       DAMO 2.3.3 (2024-04-29) tar.gz
v2.3.2       DAMO 2.3.2 (2024-04-22) tar.gz
v2.3.0       DAMO 2.3.0 (2024-04-15) tar.gz
v2.2.9       DAMO 2.2.9 (2024-04-08) tar.gz
v2.2.8       DAMO 2.2.8 (2024-04-01) tar.gz
v2.2.7       DAMO 2.2.7 (2024-03-25) tar.gz
v2.2.6       DAMO 2.2.6 (2024-03-18) tar.gz
v2.2.5       DAMO 2.2.5 (2024-03-11) tar.gz
...

# associated public inboxes:
# (number on the left is used for dev purposes)
         63 lkml
         27 linux-mm
         24 linux-doc
         14 qemu-devel
         14 u-boot
         10 linux-arm-kernel
         10 linuxppc-dev
          9 dpdk-dev
          8 netdev
          7 alsa-devel
          7 kvm
          6 git
          5 linux-wireless
          5 xen-devel
          5 linux-scsi
          5 linux-btrfs
          5 intel-gfx
          5 linux-omap
          4 linux-nfs
          4 netfilter-devel
          4 linux-fsdevel
          4 linux-devicetree
          4 linux-media
          4 stable
          4 linux-mips
          4 kvmarm
          4 linux-bluetooth
          4 dri-devel
          4 openbmc
          4 buildroot
          4 openembedded-devel
          3 linux-mtd
          3 bpf
          3 linux-mediatek
          3 linux-iommu
          3 linux-input
          3 driverdev-devel
          3 selinux
          3 linux-iio
          3 linux-gpio
          3 linux-rdma
          3 linux-ext4
          3 linux-security-module
          3 linux-ide
          3 linux-watchdog
          3 linux-trace-devel
          3 util-linux
          3 amd-gfx
          3 linux-api
          3 linux-spi
          3 linux-i2c
          3 linux-kbuild
          3 linux-arch
          3 containers
          3 linux-s390
          3 yocto-meta-freescale
          3 yocto-meta-arago
          3 outreachy
          3 intel-wired-lan
          3 poky
          3 grub-devel
          2 linux-samsung-soc
          2 linux-block
          2 linux-riscv
          2 linux-nvme
          2 linux-xfs
          2 linux-crypto
          2 linux-acpi
          2 linux-edac
          2 linux-cifs
          2 linux-arm-msm
          2 linux-amlogic
          2 linux-erofs
          2 linux-nvdimm
          2 fstests
          2 linux-pm
          2 linux-rtc
          2 linux-usb
          2 linux-renesas-soc
          2 linux-kselftest
          2 linux-clk
          2 dmaengine
          2 linux-pci
          2 linux-f2fs-devel
          2 linux-modules
          2 linux-rt-users
          2 linux-serial
          2 backports
          2 io-uring
          2 linux-man
          2 linux-mmc
          2 linux-sparse
          2 linux-tegra
          2 lttng-dev
          2 virtualization
          2 linux-fpga
          2 linux-can
          2 linux-rockchip
          2 linux-raid
          2 platform-driver-x86
          2 dm-devel
          2 linux-fbdev
          2 linux-cxl
          2 linux-perf-users
          2 lustre-devel
          2 linux-sh
          2 nouveau
          2 linux-staging
          2 linux-coco
          2 nvdimm
          2 fio
          2 iwd
          2 ltp
          2 yocto
          2 openembedded-core
          2 yocto-toaster
          2 yocto-meta-arm
          2 bitbake-devel
          2 yocto-meta-virtualization
          2 chrome-platform
          2 yocto-meta-ti
          2 openrisc
          2 kexec
          2 intel-xe
          2 linux-um
          2 virtio-comment
          2 qemu-riscv
          2 cluster-devel
          2 cgroups
          2 kvm-ppc
          2 linux-hexagon
          2 reiserfs-devel
          2 lvm-devel
          2 virtio-fs
          2 ../../../../../../../fuego
          2 ../../../../../../../igt-dev
          1 linux-fscrypt
          1 cocci
          1 live-patching
          1 linux-efi
          1 linux-integrity
          1 linux-snps-arc
          1 linux-hwmon
          1 kernel-hardening
          1 linux-next
          1 linux-parisc
          1 linux-leds
          1 kernelnewbies
          1 linux-sgx
          1 linux-hyperv
          1 workflows
          1 rcu
          1 selinux-refpolicy
          1 linux-m68k
          1 linux-i3c
          1 linux-spdx
          1 linux-wpan
          1 ksummit-discuss
          1 wireguard
          1 linux-kernel-mentees
          1 linux-csky
          1 tpmdd-devel
          1 linux-unionfs
          1 linux-firmware
          1 cip-dev
          1 linux-audit
          1 linux-remoteproc
          1 linux-dash
          1 linux-bcache
          1 mm-commits
          1 linux-pwm
          1 ceph-devel
          1 xdp-newbies
          1 phone-devel
          1 linux-hardening
          1 dash
          1 keyrings
          1 linux-sctp
          1 target-devel
          1 soc
          1 ocfs2-devel
          1 rust-for-linux
          1 ath10k
          1 ath11k
          1 linux-phy
          1 kernel-janitors
          1 sparclinux
          1 linux-sunxi
          1 mptcp
          1 regressions
          1 ksummit
          1 b43-dev
          1 linux-nfc
          1 linux-bcachefs
          1 ath9k-devel
          1 ntfs3
          1 llvm
          1 ell
          1 ofono
          1 yocto-docs
          1 linux-patches
          1 ntb
          1 xenomai
          1 damon
          1 asahi
          1 loongarch
          1 imx
          1 ath12k
          1 b4-sent
          1 linux-trace-kernel
          1 oe-linux-nfc
          1 oe-kbuild-all
          1 oe-chipsec
          1 batman
          1 virtio-dev
          1 v9fs
          1 ecryptfs
          1 linux-ia64
          1 kbd
          1 autofs
          1 cpufreq
          1 dccp
          1 devicetree-spec
          1 devicetree-compiler
          1 initramfs
          1 hail-devel
          1 kvm-ia64
          1 linux-8086
          1 kernel-testers
          1 linux-alpha
          1 linux-btrace
          1 linux-embedded
          1 linux-hams
          1 linux-hotplug
          1 linux-laptop
          1 linux-sound
          1 trinity
          1 linux-metag
          1 linux-x25
          1 linux-nilfs
          1 lvs-devel
          1 netfilter
          1 linux-oxnas
          1 u-boot-amlogic
          1 lm-sensors
          1 acpica-devel
          1 perfbook
          1 smatch
          1 ../../../../../../../wireless-regdb
          1 ../../../../../../../powertop

git clone https://yhbt.net/lore/pub/scm/linux/kernel/git/sj/damo.git