* mon config update
@ 2018-01-05 17:35 Sage Weil
2018-01-05 18:11 ` Gregory Farnum
2018-01-08 9:02 ` Piotr Dałek
0 siblings, 2 replies; 10+ messages in thread
From: Sage Weil @ 2018-01-05 17:35 UTC (permalink / raw
To: ceph-devel
My mon-based config branch is coming together. The last bit I did was the
CLI commands to set and show config. It didn't come out quite like I
expected it would. Here's what there is:
config dump Show all configuration option(s)
config get <who> {<key>} Show configuration option(s) for an entity
config rm <who> <name> Clear a configuration option for one or more
entities
config set <who> <name> <value> Cet a configuration option for one or more
entities
config show <who> Show running configuration
The show command is the oddball here: it reports what the *daemon* reports
as the running config, which includes other inputs (conf file, overrides,
etc.), while the 'get' and 'dump' are just what the mon stores. I wonder
if there is a better command than 'show'? Maybe 'running' or 'active' or
'report' or something?
A sample from my vstart cluster (vstart is only putting some of its
options in the mon so far, so this shows both conf and mon as sources):
gnit:build (wip-config) 11:28 AM $ bin/ceph config dump
WHO MASK OPTION VALUE
global crush_chooseleaf_type 0
global mon_pg_warn_min_per_osd 3
global osd_pool_default_min_size 1
global osd_pool_default_size 1
mds mds_debug_auth_pins true
mds mds_debug_frag true
mds mds_debug_subtrees true
mon mon_allow_pool_deletes true
mon mon_data_avail_crit 1
mon mon_data_avail_warn 2
mon mon_osd_reporter_subtree_level osd
mon osd_pool_default_erasure_code_profile plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd
osd osd_copyfrom_max_chunk 524288
osd osd_debug_misdirected_ops true
osd osd_debug_op_order true
osd osd_scrub_load_threshold 2000
gnit:build (wip-config) 11:28 AM $ bin/ceph config get osd.0
WHO MASK OPTION VALUE
global crush_chooseleaf_type 0
global mon_pg_warn_min_per_osd 3
osd osd_copyfrom_max_chunk 524288
osd osd_debug_misdirected_ops true
osd osd_debug_op_order true
global osd_pool_default_min_size 1
global osd_pool_default_size 1
osd osd_scrub_load_threshold 2000
gnit:build (wip-config) 11:28 AM $ bin/ceph config set osd.0 debug_osd 33
gnit:build (wip-config) 11:28 AM $ bin/ceph config get osd.0
WHO MASK OPTION VALUE
global crush_chooseleaf_type 0
osd.0 debug_osd 33/33
global mon_pg_warn_min_per_osd 3
osd osd_copyfrom_max_chunk 524288
osd osd_debug_misdirected_ops true
osd osd_debug_op_order true
global osd_pool_default_min_size 1
global osd_pool_default_size 1
osd osd_scrub_load_threshold 2000
gnit:build (wip-config) 11:28 AM $ bin/ceph config get osd.0 debug_osd
WHO MASK OPTION VALUE
osd.0 debug_osd 33/33
gnit:build (wip-config) 11:29 AM $ bin/ceph config set host:gnit debug_monc 15
gnit:build (wip-config) 11:29 AM $ bin/ceph config get osd.0
WHO MASK OPTION VALUE
global crush_chooseleaf_type 0
global host:gnit debug_monc 15/15
osd.0 debug_osd 33/33
global mon_pg_warn_min_per_osd 3
osd osd_copyfrom_max_chunk 524288
osd osd_debug_misdirected_ops true
osd osd_debug_op_order true
global osd_pool_default_min_size 1
global osd_pool_default_size 1
osd osd_scrub_load_threshold 2000
gnit:build (wip-config) 11:29 AM $ bin/ceph config show osd.0
NAME VALUE SOURCE OVERRIDES
admin_socket /tmp/ceph-asok.r90bKw/$name.asok file
bluestore_block_create 1 file
bluestore_block_db_create 1 file
bluestore_block_db_path /home/sage/src/ceph6/build/dev/osd$id/block.db.file file
bluestore_block_db_size 67108864 file
bluestore_block_wal_create 1 file
bluestore_block_wal_path /home/sage/src/ceph6/build/dev/osd$id/block.wal.file file
bluestore_block_wal_size 1048576000 file
bluestore_fsck_on_mount 1 file
chdir file
debug_bdev 20/20 file
debug_bluefs 20/20 file
debug_bluestore 30/30 file
debug_filestore 20/20 file
debug_journal 20/20 file
debug_mgrc 20/20 file
debug_monc 20/20 file
debug_ms 1/1 file
debug_objclass 20/20 file
debug_objecter 20/20 file
debug_osd 25/25 file mon
debug_reserver 10/10 file
debug_rocksdb 10/10 file
enable_experimental_unrecoverable_data_corrupting_features * file
erasure_code_dir /home/sage/src/ceph6/build/lib file
filestore_fd_cache_size 32 file
filestore_wbthrottle_btrfs_inodes_hard_limit 30 file
filestore_wbthrottle_btrfs_ios_hard_limit 20 file
filestore_wbthrottle_btrfs_ios_start_flusher 10 file
filestore_wbthrottle_xfs_inodes_hard_limit 30 file
filestore_wbthrottle_xfs_ios_hard_limit 20 file
filestore_wbthrottle_xfs_ios_start_flusher 10 file
heartbeat_file /home/sage/src/ceph6/build/out/$name.heartbeat file
keyring $osd_data/keyring default
leveldb_log default
lockdep 1 file
log_file /home/sage/src/ceph6/build/out/$name.log file
mon_osd_backfillfull_ratio 0.99 file
mon_osd_full_ratio 0.99 file
mon_osd_nearfull_ratio 0.99 file
mon_pg_warn_min_per_osd 3 mon
osd_check_max_object_name_len_on_startup 0 file
osd_class_default_list * file
osd_class_dir /home/sage/src/ceph6/build/lib file
osd_class_load_list * file
osd_copyfrom_max_chunk 524288 mon
osd_data /home/sage/src/ceph6/build/dev/osd$id file
osd_debug_misdirected_ops 1 mon
osd_debug_op_order 1 mon
osd_failsafe_full_ratio 0.99 file
osd_journal /home/sage/src/ceph6/build/dev/osd$id/journal file
osd_journal_size 100 file
osd_objectstore bluestore override file
osd_pool_default_min_size 1 mon
osd_pool_default_size 1 mon
osd_scrub_load_threshold 2000 mon
pid_file /home/sage/src/ceph6/build/out/$name.pid file
plugin_dir /home/sage/src/ceph6/build/lib file
run_dir /home/sage/src/ceph6/build/out file
gnit:build (wip-config) 11:29 AM $ bin/ceph config set osd/host:gnit debug_filestore 20
gnit:build (wip-config) 11:32 AM $ bin/ceph config get osd.0
WHO MASK OPTION VALUE
global crush_chooseleaf_type 0
osd host:gnit debug_filestore 20/20
global host:gnit debug_monc 15/15
osd.0 debug_osd 33/33
global mon_pg_warn_min_per_osd 3
osd osd_copyfrom_max_chunk 524288
osd osd_debug_misdirected_ops true
osd osd_debug_op_order true
global osd_pool_default_min_size 1
global osd_pool_default_size 1
osd osd_scrub_load_threshold 2000
gnit:build (wip-config) 11:32 AM $ bin/ceph config set osd/class:ssd debug_bluestore 0
gnit:build (wip-config) 11:33 AM $ bin/ceph config set osd/class:hdd debug_bluestore 20
gnit:build (wip-config) 11:33 AM $ bin/ceph config get osd.0
WHO MASK OPTION VALUE
global crush_chooseleaf_type 0
osd class:ssd debug_bluestore 0/0
osd host:gnit debug_filestore 20/20
global host:gnit debug_monc 15/15
osd.0 debug_osd 33/33
global mon_pg_warn_min_per_osd 3
osd osd_copyfrom_max_chunk 524288
osd osd_debug_misdirected_ops true
osd osd_debug_op_order true
global osd_pool_default_min_size 1
global osd_pool_default_size 1
osd osd_scrub_load_threshold 2000
(osd.0 is an ssd)
Thoughts?
sage
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: mon config update
2018-01-05 17:35 mon config update Sage Weil
@ 2018-01-05 18:11 ` Gregory Farnum
2018-01-05 18:24 ` Sage Weil
2018-01-08 9:02 ` Piotr Dałek
1 sibling, 1 reply; 10+ messages in thread
From: Gregory Farnum @ 2018-01-05 18:11 UTC (permalink / raw
To: Sage Weil; +Cc: ceph-devel
On Fri, Jan 5, 2018 at 9:35 AM, Sage Weil <sweil@redhat.com> wrote:
> My mon-based config branch is coming together. The last bit I did was the
> CLI commands to set and show config. It didn't come out quite like I
> expected it would. Here's what there is:
>
> config dump Show all configuration option(s)
> config get <who> {<key>} Show configuration option(s) for an entity
> config rm <who> <name> Clear a configuration option for one or more
> entities
> config set <who> <name> <value> Cet a configuration option for one or more
> entities
> config show <who> Show running configuration
>
> The show command is the oddball here: it reports what the *daemon* reports
> as the running config, which includes other inputs (conf file, overrides,
> etc.), while the 'get' and 'dump' are just what the mon stores. I wonder
> if there is a better command than 'show'? Maybe 'running' or 'active' or
> 'report' or something?
I don't have a good suggestion for this name, but I was a little
puzzled until I realized it seems to only show the options which are
non-default? Given that defaults can change across software versions,
I'm not sure that's a feasible long-term strategy. (In addition to
dealing with whether we're comparing against the defaults on the OSD
version or the monitor version, if I get a dump of options from
somebody to do debugging with I'm not going to remember that a timeout
changed between point releases and they're operating differently!)
Although I definitely get the desire to make the config legible when
dumped, I'm not sure there's much utility there as a default anyway —
won't admins mostly be querying for a specific value or else running
diffs between different dumps? Perhaps we can deal with the bad naming
by having "ceph config show non-default <who>", "ceph config show
current <who>", and whatever else we like. (Maybe options for showing
configs under USER, ADVANCED, and DEV?)
Looking at the output, the WHO versus MASK bit is a little thorny. Can
you show what it looks like when there are multiple masks on the same
option?
It looks like things are dumped in config declaration order, right?
Given that, maybe OPTION should be the "top" (leftmost) column and
then WHO and MASK can be filed underneath it, with nicely indented
output to make it more like a tree.
Or else make it more like a tree in the other direction, with either
WHO or MASK as the root and with output order controlled that way. But
that probably involved a lot more repetition of the output.
-Greg
>
> A sample from my vstart cluster (vstart is only putting some of its
> options in the mon so far, so this shows both conf and mon as sources):
>
> gnit:build (wip-config) 11:28 AM $ bin/ceph config dump
> WHO MASK OPTION VALUE
> global crush_chooseleaf_type 0
> global mon_pg_warn_min_per_osd 3
> global osd_pool_default_min_size 1
> global osd_pool_default_size 1
> mds mds_debug_auth_pins true
> mds mds_debug_frag true
> mds mds_debug_subtrees true
> mon mon_allow_pool_deletes true
> mon mon_data_avail_crit 1
> mon mon_data_avail_warn 2
> mon mon_osd_reporter_subtree_level osd
> mon osd_pool_default_erasure_code_profile plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd
> osd osd_copyfrom_max_chunk 524288
> osd osd_debug_misdirected_ops true
> osd osd_debug_op_order true
> osd osd_scrub_load_threshold 2000
> gnit:build (wip-config) 11:28 AM $ bin/ceph config get osd.0
> WHO MASK OPTION VALUE
> global crush_chooseleaf_type 0
> global mon_pg_warn_min_per_osd 3
> osd osd_copyfrom_max_chunk 524288
> osd osd_debug_misdirected_ops true
> osd osd_debug_op_order true
> global osd_pool_default_min_size 1
> global osd_pool_default_size 1
> osd osd_scrub_load_threshold 2000
> gnit:build (wip-config) 11:28 AM $ bin/ceph config set osd.0 debug_osd 33
> gnit:build (wip-config) 11:28 AM $ bin/ceph config get osd.0
> WHO MASK OPTION VALUE
> global crush_chooseleaf_type 0
> osd.0 debug_osd 33/33
> global mon_pg_warn_min_per_osd 3
> osd osd_copyfrom_max_chunk 524288
> osd osd_debug_misdirected_ops true
> osd osd_debug_op_order true
> global osd_pool_default_min_size 1
> global osd_pool_default_size 1
> osd osd_scrub_load_threshold 2000
> gnit:build (wip-config) 11:28 AM $ bin/ceph config get osd.0 debug_osd
> WHO MASK OPTION VALUE
> osd.0 debug_osd 33/33
> gnit:build (wip-config) 11:29 AM $ bin/ceph config set host:gnit debug_monc 15
> gnit:build (wip-config) 11:29 AM $ bin/ceph config get osd.0
> WHO MASK OPTION VALUE
> global crush_chooseleaf_type 0
> global host:gnit debug_monc 15/15
> osd.0 debug_osd 33/33
> global mon_pg_warn_min_per_osd 3
> osd osd_copyfrom_max_chunk 524288
> osd osd_debug_misdirected_ops true
> osd osd_debug_op_order true
> global osd_pool_default_min_size 1
> global osd_pool_default_size 1
> osd osd_scrub_load_threshold 2000
> gnit:build (wip-config) 11:29 AM $ bin/ceph config show osd.0
> NAME VALUE SOURCE OVERRIDES
> admin_socket /tmp/ceph-asok.r90bKw/$name.asok file
> bluestore_block_create 1 file
> bluestore_block_db_create 1 file
> bluestore_block_db_path /home/sage/src/ceph6/build/dev/osd$id/block.db.file file
> bluestore_block_db_size 67108864 file
> bluestore_block_wal_create 1 file
> bluestore_block_wal_path /home/sage/src/ceph6/build/dev/osd$id/block.wal.file file
> bluestore_block_wal_size 1048576000 file
> bluestore_fsck_on_mount 1 file
> chdir file
> debug_bdev 20/20 file
> debug_bluefs 20/20 file
> debug_bluestore 30/30 file
> debug_filestore 20/20 file
> debug_journal 20/20 file
> debug_mgrc 20/20 file
> debug_monc 20/20 file
> debug_ms 1/1 file
> debug_objclass 20/20 file
> debug_objecter 20/20 file
> debug_osd 25/25 file mon
> debug_reserver 10/10 file
> debug_rocksdb 10/10 file
> enable_experimental_unrecoverable_data_corrupting_features * file
> erasure_code_dir /home/sage/src/ceph6/build/lib file
> filestore_fd_cache_size 32 file
> filestore_wbthrottle_btrfs_inodes_hard_limit 30 file
> filestore_wbthrottle_btrfs_ios_hard_limit 20 file
> filestore_wbthrottle_btrfs_ios_start_flusher 10 file
> filestore_wbthrottle_xfs_inodes_hard_limit 30 file
> filestore_wbthrottle_xfs_ios_hard_limit 20 file
> filestore_wbthrottle_xfs_ios_start_flusher 10 file
> heartbeat_file /home/sage/src/ceph6/build/out/$name.heartbeat file
> keyring $osd_data/keyring default
> leveldb_log default
> lockdep 1 file
> log_file /home/sage/src/ceph6/build/out/$name.log file
> mon_osd_backfillfull_ratio 0.99 file
> mon_osd_full_ratio 0.99 file
> mon_osd_nearfull_ratio 0.99 file
> mon_pg_warn_min_per_osd 3 mon
> osd_check_max_object_name_len_on_startup 0 file
> osd_class_default_list * file
> osd_class_dir /home/sage/src/ceph6/build/lib file
> osd_class_load_list * file
> osd_copyfrom_max_chunk 524288 mon
> osd_data /home/sage/src/ceph6/build/dev/osd$id file
> osd_debug_misdirected_ops 1 mon
> osd_debug_op_order 1 mon
> osd_failsafe_full_ratio 0.99 file
> osd_journal /home/sage/src/ceph6/build/dev/osd$id/journal file
> osd_journal_size 100 file
> osd_objectstore bluestore override file
> osd_pool_default_min_size 1 mon
> osd_pool_default_size 1 mon
> osd_scrub_load_threshold 2000 mon
> pid_file /home/sage/src/ceph6/build/out/$name.pid file
> plugin_dir /home/sage/src/ceph6/build/lib file
> run_dir /home/sage/src/ceph6/build/out file
> gnit:build (wip-config) 11:29 AM $ bin/ceph config set osd/host:gnit debug_filestore 20
> gnit:build (wip-config) 11:32 AM $ bin/ceph config get osd.0
> WHO MASK OPTION VALUE
> global crush_chooseleaf_type 0
> osd host:gnit debug_filestore 20/20
> global host:gnit debug_monc 15/15
> osd.0 debug_osd 33/33
> global mon_pg_warn_min_per_osd 3
> osd osd_copyfrom_max_chunk 524288
> osd osd_debug_misdirected_ops true
> osd osd_debug_op_order true
> global osd_pool_default_min_size 1
> global osd_pool_default_size 1
> osd osd_scrub_load_threshold 2000
> gnit:build (wip-config) 11:32 AM $ bin/ceph config set osd/class:ssd debug_bluestore 0
> gnit:build (wip-config) 11:33 AM $ bin/ceph config set osd/class:hdd debug_bluestore 20
> gnit:build (wip-config) 11:33 AM $ bin/ceph config get osd.0
> WHO MASK OPTION VALUE
> global crush_chooseleaf_type 0
> osd class:ssd debug_bluestore 0/0
> osd host:gnit debug_filestore 20/20
> global host:gnit debug_monc 15/15
> osd.0 debug_osd 33/33
> global mon_pg_warn_min_per_osd 3
> osd osd_copyfrom_max_chunk 524288
> osd osd_debug_misdirected_ops true
> osd osd_debug_op_order true
> global osd_pool_default_min_size 1
> global osd_pool_default_size 1
> osd osd_scrub_load_threshold 2000
>
> (osd.0 is an ssd)
>
> Thoughts?
> sage
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: mon config update
2018-01-05 18:11 ` Gregory Farnum
@ 2018-01-05 18:24 ` Sage Weil
2018-01-05 18:39 ` Gregory Farnum
0 siblings, 1 reply; 10+ messages in thread
From: Sage Weil @ 2018-01-05 18:24 UTC (permalink / raw
To: Gregory Farnum; +Cc: ceph-devel
[-- Attachment #1: Type: TEXT/PLAIN, Size: 20747 bytes --]
On Fri, 5 Jan 2018, Gregory Farnum wrote:
> On Fri, Jan 5, 2018 at 9:35 AM, Sage Weil <sweil@redhat.com> wrote:
> > My mon-based config branch is coming together. The last bit I did was the
> > CLI commands to set and show config. It didn't come out quite like I
> > expected it would. Here's what there is:
> >
> > config dump Show all configuration option(s)
> > config get <who> {<key>} Show configuration option(s) for an entity
> > config rm <who> <name> Clear a configuration option for one or more
> > entities
> > config set <who> <name> <value> Cet a configuration option for one or more
> > entities
> > config show <who> Show running configuration
> >
> > The show command is the oddball here: it reports what the *daemon* reports
> > as the running config, which includes other inputs (conf file, overrides,
> > etc.), while the 'get' and 'dump' are just what the mon stores. I wonder
> > if there is a better command than 'show'? Maybe 'running' or 'active' or
> > 'report' or something?
>
> I don't have a good suggestion for this name, but I was a little
> puzzled until I realized it seems to only show the options which are
> non-default? Given that defaults can change across software versions,
> I'm not sure that's a feasible long-term strategy. (In addition to
> dealing with whether we're comparing against the defaults on the OSD
> version or the monitor version, if I get a dump of options from
> somebody to do debugging with I'm not going to remember that a timeout
> changed between point releases and they're operating differently!)
>
> Although I definitely get the desire to make the config legible when
> dumped, I'm not sure there's much utility there as a default anyway —
> won't admins mostly be querying for a specific value or else running
> diffs between different dumps? Perhaps we can deal with the bad naming
> by having "ceph config show non-default <who>", "ceph config show
> current <who>", and whatever else we like.
Yeah, defaults are excluded because there are 1000+ options and the output
becomes unusable. I'd prefer to make the less-standard command the one
that dumps everything (including defaults), like show-all, or similar.
Note though that currently the daemons aren't reporting their default
values to the mgr at all--this keeps the MMgrReport message smaller.
> (Maybe options for showing configs under USER, ADVANCED, and DEV?)
Oh yeah, I forgot about these! I guess I should add columns for these for
both 'dump' and 'get'?
Also, I guess the other main missing piece is 'ceph config help <option>',
which should present a friendly summary of the option, its description,
defaults, and so on?
> Looking at the output, the WHO versus MASK bit is a little thorny. Can
> you show what it looks like when there are multiple masks on the same
> option?
gnit:build (wip-config) 12:15 PM $ bin/ceph config set osd/class:ssd/host:gnit debug_bluestore 20
gnit:build (wip-config) 12:15 PM $ bin/ceph config get osd.0 debug_bluestore
WHO MASK OPTION VALUE
osd host:gnit/class:ssd debug_bluestore 20/20
gnit:build (wip-config) 12:15 PM $ bin/ceph config set osd.0/class:ssd/host:gnit debug_bluestore 21
gnit:build (wip-config) 12:15 PM $ bin/ceph config get osd.0 debug_bluestore
WHO MASK OPTION VALUE
osd.0 host:gnit/class:ssd debug_bluestore 21/21
> It looks like things are dumped in config declaration order, right?
> Given that, maybe OPTION should be the "top" (leftmost) column and
> then WHO and MASK can be filed underneath it, with nicely indented
> output to make it more like a tree.
> Or else make it more like a tree in the other direction, with either
> WHO or MASK as the root and with output order controlled that way. But
> that probably involved a lot more repetition of the output.
They're sorted by section (global, type, daemon), then by option name.
Indenting is a great idea... I'd probably indent just the 'who' portion,
like so? Is that what you mean?
WHO MASK OPTION VALUE
global crush_chooseleaf_type 0
global mon_pg_warn_min_per_osd 3
global osd_pool_default_min_size 1
global osd_pool_default_size 1
client admin_socket /tmp/ceph-asok.A5CIZs/$name.$pid.asok
client log_file /home/sage/src/ceph6/build/out/$name.$pid.log
client.rgw rgw_crypt_require_ssl false
client.rgw rgw_crypt_s3_kms_encryption_keys testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo=
client.rgw rgw_frontends civetweb port=8000
client.rgw rgw_lc_debug_interval 10
mds debug_mds 20/20
mds debug_mgrc 20/20
osd bluestore_block_create true
osd bluestore_block_db_create true
osd debug_bluestore 30/30
osd host:gnit/class:hdd debug_bluestore 20/20
osd host:gnit/class:ssd debug_bluestore 20/20
osd.0 host:gnit/class:ssd debug_bluestore 21/21
I'm not sure sorting by the option name would work well here since you'd
have, say, all of the debug_* options grouped together but applying to
different things. The idea was to easily see what the osd configs are,
similar to what you get when looking at the old ini file. And then also
see any specializations per daemon below it.
sage
> -Greg
>
> >
> > A sample from my vstart cluster (vstart is only putting some of its
> > options in the mon so far, so this shows both conf and mon as sources):
> >
> > gnit:build (wip-config) 11:28 AM $ bin/ceph config dump
> > WHO MASK OPTION VALUE
> > global crush_chooseleaf_type 0
> > global mon_pg_warn_min_per_osd 3
> > global osd_pool_default_min_size 1
> > global osd_pool_default_size 1
> > mds mds_debug_auth_pins true
> > mds mds_debug_frag true
> > mds mds_debug_subtrees true
> > mon mon_allow_pool_deletes true
> > mon mon_data_avail_crit 1
> > mon mon_data_avail_warn 2
> > mon mon_osd_reporter_subtree_level osd
> > mon osd_pool_default_erasure_code_profile plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd
> > osd osd_copyfrom_max_chunk 524288
> > osd osd_debug_misdirected_ops true
> > osd osd_debug_op_order true
> > osd osd_scrub_load_threshold 2000
> > gnit:build (wip-config) 11:28 AM $ bin/ceph config get osd.0
> > WHO MASK OPTION VALUE
> > global crush_chooseleaf_type 0
> > global mon_pg_warn_min_per_osd 3
> > osd osd_copyfrom_max_chunk 524288
> > osd osd_debug_misdirected_ops true
> > osd osd_debug_op_order true
> > global osd_pool_default_min_size 1
> > global osd_pool_default_size 1
> > osd osd_scrub_load_threshold 2000
> > gnit:build (wip-config) 11:28 AM $ bin/ceph config set osd.0 debug_osd 33
> > gnit:build (wip-config) 11:28 AM $ bin/ceph config get osd.0
> > WHO MASK OPTION VALUE
> > global crush_chooseleaf_type 0
> > osd.0 debug_osd 33/33
> > global mon_pg_warn_min_per_osd 3
> > osd osd_copyfrom_max_chunk 524288
> > osd osd_debug_misdirected_ops true
> > osd osd_debug_op_order true
> > global osd_pool_default_min_size 1
> > global osd_pool_default_size 1
> > osd osd_scrub_load_threshold 2000
> > gnit:build (wip-config) 11:28 AM $ bin/ceph config get osd.0 debug_osd
> > WHO MASK OPTION VALUE
> > osd.0 debug_osd 33/33
> > gnit:build (wip-config) 11:29 AM $ bin/ceph config set host:gnit debug_monc 15
> > gnit:build (wip-config) 11:29 AM $ bin/ceph config get osd.0
> > WHO MASK OPTION VALUE
> > global crush_chooseleaf_type 0
> > global host:gnit debug_monc 15/15
> > osd.0 debug_osd 33/33
> > global mon_pg_warn_min_per_osd 3
> > osd osd_copyfrom_max_chunk 524288
> > osd osd_debug_misdirected_ops true
> > osd osd_debug_op_order true
> > global osd_pool_default_min_size 1
> > global osd_pool_default_size 1
> > osd osd_scrub_load_threshold 2000
> > gnit:build (wip-config) 11:29 AM $ bin/ceph config show osd.0
> > NAME VALUE SOURCE OVERRIDES
> > admin_socket /tmp/ceph-asok.r90bKw/$name.asok file
> > bluestore_block_create 1 file
> > bluestore_block_db_create 1 file
> > bluestore_block_db_path /home/sage/src/ceph6/build/dev/osd$id/block.db.file file
> > bluestore_block_db_size 67108864 file
> > bluestore_block_wal_create 1 file
> > bluestore_block_wal_path /home/sage/src/ceph6/build/dev/osd$id/block.wal.file file
> > bluestore_block_wal_size 1048576000 file
> > bluestore_fsck_on_mount 1 file
> > chdir file
> > debug_bdev 20/20 file
> > debug_bluefs 20/20 file
> > debug_bluestore 30/30 file
> > debug_filestore 20/20 file
> > debug_journal 20/20 file
> > debug_mgrc 20/20 file
> > debug_monc 20/20 file
> > debug_ms 1/1 file
> > debug_objclass 20/20 file
> > debug_objecter 20/20 file
> > debug_osd 25/25 file mon
> > debug_reserver 10/10 file
> > debug_rocksdb 10/10 file
> > enable_experimental_unrecoverable_data_corrupting_features * file
> > erasure_code_dir /home/sage/src/ceph6/build/lib file
> > filestore_fd_cache_size 32 file
> > filestore_wbthrottle_btrfs_inodes_hard_limit 30 file
> > filestore_wbthrottle_btrfs_ios_hard_limit 20 file
> > filestore_wbthrottle_btrfs_ios_start_flusher 10 file
> > filestore_wbthrottle_xfs_inodes_hard_limit 30 file
> > filestore_wbthrottle_xfs_ios_hard_limit 20 file
> > filestore_wbthrottle_xfs_ios_start_flusher 10 file
> > heartbeat_file /home/sage/src/ceph6/build/out/$name.heartbeat file
> > keyring $osd_data/keyring default
> > leveldb_log default
> > lockdep 1 file
> > log_file /home/sage/src/ceph6/build/out/$name.log file
> > mon_osd_backfillfull_ratio 0.99 file
> > mon_osd_full_ratio 0.99 file
> > mon_osd_nearfull_ratio 0.99 file
> > mon_pg_warn_min_per_osd 3 mon
> > osd_check_max_object_name_len_on_startup 0 file
> > osd_class_default_list * file
> > osd_class_dir /home/sage/src/ceph6/build/lib file
> > osd_class_load_list * file
> > osd_copyfrom_max_chunk 524288 mon
> > osd_data /home/sage/src/ceph6/build/dev/osd$id file
> > osd_debug_misdirected_ops 1 mon
> > osd_debug_op_order 1 mon
> > osd_failsafe_full_ratio 0.99 file
> > osd_journal /home/sage/src/ceph6/build/dev/osd$id/journal file
> > osd_journal_size 100 file
> > osd_objectstore bluestore override file
> > osd_pool_default_min_size 1 mon
> > osd_pool_default_size 1 mon
> > osd_scrub_load_threshold 2000 mon
> > pid_file /home/sage/src/ceph6/build/out/$name.pid file
> > plugin_dir /home/sage/src/ceph6/build/lib file
> > run_dir /home/sage/src/ceph6/build/out file
> > gnit:build (wip-config) 11:29 AM $ bin/ceph config set osd/host:gnit debug_filestore 20
> > gnit:build (wip-config) 11:32 AM $ bin/ceph config get osd.0
> > WHO MASK OPTION VALUE
> > global crush_chooseleaf_type 0
> > osd host:gnit debug_filestore 20/20
> > global host:gnit debug_monc 15/15
> > osd.0 debug_osd 33/33
> > global mon_pg_warn_min_per_osd 3
> > osd osd_copyfrom_max_chunk 524288
> > osd osd_debug_misdirected_ops true
> > osd osd_debug_op_order true
> > global osd_pool_default_min_size 1
> > global osd_pool_default_size 1
> > osd osd_scrub_load_threshold 2000
> > gnit:build (wip-config) 11:32 AM $ bin/ceph config set osd/class:ssd debug_bluestore 0
> > gnit:build (wip-config) 11:33 AM $ bin/ceph config set osd/class:hdd debug_bluestore 20
> > gnit:build (wip-config) 11:33 AM $ bin/ceph config get osd.0
> > WHO MASK OPTION VALUE
> > global crush_chooseleaf_type 0
> > osd class:ssd debug_bluestore 0/0
> > osd host:gnit debug_filestore 20/20
> > global host:gnit debug_monc 15/15
> > osd.0 debug_osd 33/33
> > global mon_pg_warn_min_per_osd 3
> > osd osd_copyfrom_max_chunk 524288
> > osd osd_debug_misdirected_ops true
> > osd osd_debug_op_order true
> > global osd_pool_default_min_size 1
> > global osd_pool_default_size 1
> > osd osd_scrub_load_threshold 2000
> >
> > (osd.0 is an ssd)
> >
> > Thoughts?
> > sage
> > --
> > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at http://vger.kernel.org/majordomo-info.html
>
>
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: mon config update
2018-01-05 18:24 ` Sage Weil
@ 2018-01-05 18:39 ` Gregory Farnum
2018-01-05 21:00 ` Sage Weil
0 siblings, 1 reply; 10+ messages in thread
From: Gregory Farnum @ 2018-01-05 18:39 UTC (permalink / raw
To: Sage Weil; +Cc: ceph-devel
On Fri, Jan 5, 2018 at 10:24 AM, Sage Weil <sweil@redhat.com> wrote:
> On Fri, 5 Jan 2018, Gregory Farnum wrote:
>> On Fri, Jan 5, 2018 at 9:35 AM, Sage Weil <sweil@redhat.com> wrote:
>> > My mon-based config branch is coming together. The last bit I did was the
>> > CLI commands to set and show config. It didn't come out quite like I
>> > expected it would. Here's what there is:
>> >
>> > config dump Show all configuration option(s)
>> > config get <who> {<key>} Show configuration option(s) for an entity
>> > config rm <who> <name> Clear a configuration option for one or more
>> > entities
>> > config set <who> <name> <value> Cet a configuration option for one or more
>> > entities
>> > config show <who> Show running configuration
>> >
>> > The show command is the oddball here: it reports what the *daemon* reports
>> > as the running config, which includes other inputs (conf file, overrides,
>> > etc.), while the 'get' and 'dump' are just what the mon stores. I wonder
>> > if there is a better command than 'show'? Maybe 'running' or 'active' or
>> > 'report' or something?
>>
>> I don't have a good suggestion for this name, but I was a little
>> puzzled until I realized it seems to only show the options which are
>> non-default? Given that defaults can change across software versions,
>> I'm not sure that's a feasible long-term strategy. (In addition to
>> dealing with whether we're comparing against the defaults on the OSD
>> version or the monitor version, if I get a dump of options from
>> somebody to do debugging with I'm not going to remember that a timeout
>> changed between point releases and they're operating differently!)
>>
>> Although I definitely get the desire to make the config legible when
>> dumped, I'm not sure there's much utility there as a default anyway —
>> won't admins mostly be querying for a specific value or else running
>> diffs between different dumps? Perhaps we can deal with the bad naming
>> by having "ceph config show non-default <who>", "ceph config show
>> current <who>", and whatever else we like.
>
> Yeah, defaults are excluded because there are 1000+ options and the output
> becomes unusable. I'd prefer to make the less-standard command the one
> that dumps everything (including defaults), like show-all, or similar.
> Note though that currently the daemons aren't reporting their default
> values to the mgr at all--this keeps the MMgrReport message smaller.
Hmm. Like I said, this is going to kill somebody on debugging someday.
I think we just need to eat it. (We could only send the config options
when one changes, if we're worried about their size or parsing cost.)
>
>> (Maybe options for showing configs under USER, ADVANCED, and DEV?)
>
> Oh yeah, I forgot about these! I guess I should add columns for these for
> both 'dump' and 'get'?
>
> Also, I guess the other main missing piece is 'ceph config help <option>',
> which should present a friendly summary of the option, its description,
> defaults, and so on?
>
>> Looking at the output, the WHO versus MASK bit is a little thorny. Can
>> you show what it looks like when there are multiple masks on the same
>> option?
>
> gnit:build (wip-config) 12:15 PM $ bin/ceph config set osd/class:ssd/host:gnit debug_bluestore 20
> gnit:build (wip-config) 12:15 PM $ bin/ceph config get osd.0 debug_bluestore
> WHO MASK OPTION VALUE
> osd host:gnit/class:ssd debug_bluestore 20/20
> gnit:build (wip-config) 12:15 PM $ bin/ceph config set osd.0/class:ssd/host:gnit debug_bluestore 21
> gnit:build (wip-config) 12:15 PM $ bin/ceph config get osd.0 debug_bluestore
> WHO MASK OPTION VALUE
> osd.0 host:gnit/class:ssd debug_bluestore 21/21
I was unclear. I mean, what's it look like when there's a setting for
osd/class:ssd and a different setting for osd/class:hdd? ...which I
now realize you did have, but I missed it and that's a good
illustration for what I meant. See end of email.
>> It looks like things are dumped in config declaration order, right?
>> Given that, maybe OPTION should be the "top" (leftmost) column and
>> then WHO and MASK can be filed underneath it, with nicely indented
>> output to make it more like a tree.
>> Or else make it more like a tree in the other direction, with either
>> WHO or MASK as the root and with output order controlled that way. But
>> that probably involved a lot more repetition of the output.
>
> They're sorted by section (global, type, daemon), then by option name.
> Indenting is a great idea... I'd probably indent just the 'who' portion,
> like so? Is that what you mean?
>
> WHO MASK OPTION VALUE
> global crush_chooseleaf_type 0
> global mon_pg_warn_min_per_osd 3
> global osd_pool_default_min_size 1
> global osd_pool_default_size 1
> client admin_socket /tmp/ceph-asok.A5CIZs/$name.$pid.asok
> client log_file /home/sage/src/ceph6/build/out/$name.$pid.log
> client.rgw rgw_crypt_require_ssl false
> client.rgw rgw_crypt_s3_kms_encryption_keys testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo=
> client.rgw rgw_frontends civetweb port=8000
> client.rgw rgw_lc_debug_interval 10
So that's not what I meant, but I like it if we're going this way.
> mds debug_mds 20/20
> mds debug_mgrc 20/20
> osd bluestore_block_create true
> osd bluestore_block_db_create true
> osd debug_bluestore 30/30
> osd host:gnit/class:hdd debug_bluestore 20/20
> osd host:gnit/class:ssd debug_bluestore 20/20
> osd.0 host:gnit/class:ssd debug_bluestore 21/21
>
> I'm not sure sorting by the option name would work well here since you'd
> have, say, all of the debug_* options grouped together but applying to
> different things. The idea was to easily see what the osd configs are,
> similar to what you get when looking at the old ini file. And then also
> see any specializations per daemon below it.
Here's (a bad manually-typed illustration of) what I was suggesting:
OPTION WHO MASK
VALUE
log_file client
/tmp/ceph-asok.A5CIZs/$name.$pid.asok
mds
/tmp/ceph-asok.A5CIZs/mds.$id.$pid.asok
host:gnit
/tmp/ceph-asok.special_gnit/mds.$id.$pid.asok
bluestore_block_create osd
true
bluestore_block_db_create osd
true
debug_bluestore osd
host:gnit/class:hdd 20/20
host:gnit/class:ssd 20/20
osd.0
host:gnit/class:ssd 21/21
debug_filestore osd
10/10
So this way the options which have different settings can be visually
identified; and the same for masks versus daemon-type. (Although I'm
not sure which order we want those columns to go in.)
-Greg
>
> sage
>
>
>> -Greg
>>
>> >
>> > A sample from my vstart cluster (vstart is only putting some of its
>> > options in the mon so far, so this shows both conf and mon as sources):
>> >
>> > gnit:build (wip-config) 11:28 AM $ bin/ceph config dump
>> > WHO MASK OPTION VALUE
>> > global crush_chooseleaf_type 0
>> > global mon_pg_warn_min_per_osd 3
>> > global osd_pool_default_min_size 1
>> > global osd_pool_default_size 1
>> > mds mds_debug_auth_pins true
>> > mds mds_debug_frag true
>> > mds mds_debug_subtrees true
>> > mon mon_allow_pool_deletes true
>> > mon mon_data_avail_crit 1
>> > mon mon_data_avail_warn 2
>> > mon mon_osd_reporter_subtree_level osd
>> > mon osd_pool_default_erasure_code_profile plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd
>> > osd osd_copyfrom_max_chunk 524288
>> > osd osd_debug_misdirected_ops true
>> > osd osd_debug_op_order true
>> > osd osd_scrub_load_threshold 2000
>> > gnit:build (wip-config) 11:28 AM $ bin/ceph config get osd.0
>> > WHO MASK OPTION VALUE
>> > global crush_chooseleaf_type 0
>> > global mon_pg_warn_min_per_osd 3
>> > osd osd_copyfrom_max_chunk 524288
>> > osd osd_debug_misdirected_ops true
>> > osd osd_debug_op_order true
>> > global osd_pool_default_min_size 1
>> > global osd_pool_default_size 1
>> > osd osd_scrub_load_threshold 2000
>> > gnit:build (wip-config) 11:28 AM $ bin/ceph config set osd.0 debug_osd 33
>> > gnit:build (wip-config) 11:28 AM $ bin/ceph config get osd.0
>> > WHO MASK OPTION VALUE
>> > global crush_chooseleaf_type 0
>> > osd.0 debug_osd 33/33
>> > global mon_pg_warn_min_per_osd 3
>> > osd osd_copyfrom_max_chunk 524288
>> > osd osd_debug_misdirected_ops true
>> > osd osd_debug_op_order true
>> > global osd_pool_default_min_size 1
>> > global osd_pool_default_size 1
>> > osd osd_scrub_load_threshold 2000
>> > gnit:build (wip-config) 11:28 AM $ bin/ceph config get osd.0 debug_osd
>> > WHO MASK OPTION VALUE
>> > osd.0 debug_osd 33/33
>> > gnit:build (wip-config) 11:29 AM $ bin/ceph config set host:gnit debug_monc 15
>> > gnit:build (wip-config) 11:29 AM $ bin/ceph config get osd.0
>> > WHO MASK OPTION VALUE
>> > global crush_chooseleaf_type 0
>> > global host:gnit debug_monc 15/15
>> > osd.0 debug_osd 33/33
>> > global mon_pg_warn_min_per_osd 3
>> > osd osd_copyfrom_max_chunk 524288
>> > osd osd_debug_misdirected_ops true
>> > osd osd_debug_op_order true
>> > global osd_pool_default_min_size 1
>> > global osd_pool_default_size 1
>> > osd osd_scrub_load_threshold 2000
>> > gnit:build (wip-config) 11:29 AM $ bin/ceph config show osd.0
>> > NAME VALUE SOURCE OVERRIDES
>> > admin_socket /tmp/ceph-asok.r90bKw/$name.asok file
>> > bluestore_block_create 1 file
>> > bluestore_block_db_create 1 file
>> > bluestore_block_db_path /home/sage/src/ceph6/build/dev/osd$id/block.db.file file
>> > bluestore_block_db_size 67108864 file
>> > bluestore_block_wal_create 1 file
>> > bluestore_block_wal_path /home/sage/src/ceph6/build/dev/osd$id/block.wal.file file
>> > bluestore_block_wal_size 1048576000 file
>> > bluestore_fsck_on_mount 1 file
>> > chdir file
>> > debug_bdev 20/20 file
>> > debug_bluefs 20/20 file
>> > debug_bluestore 30/30 file
>> > debug_filestore 20/20 file
>> > debug_journal 20/20 file
>> > debug_mgrc 20/20 file
>> > debug_monc 20/20 file
>> > debug_ms 1/1 file
>> > debug_objclass 20/20 file
>> > debug_objecter 20/20 file
>> > debug_osd 25/25 file mon
>> > debug_reserver 10/10 file
>> > debug_rocksdb 10/10 file
>> > enable_experimental_unrecoverable_data_corrupting_features * file
>> > erasure_code_dir /home/sage/src/ceph6/build/lib file
>> > filestore_fd_cache_size 32 file
>> > filestore_wbthrottle_btrfs_inodes_hard_limit 30 file
>> > filestore_wbthrottle_btrfs_ios_hard_limit 20 file
>> > filestore_wbthrottle_btrfs_ios_start_flusher 10 file
>> > filestore_wbthrottle_xfs_inodes_hard_limit 30 file
>> > filestore_wbthrottle_xfs_ios_hard_limit 20 file
>> > filestore_wbthrottle_xfs_ios_start_flusher 10 file
>> > heartbeat_file /home/sage/src/ceph6/build/out/$name.heartbeat file
>> > keyring $osd_data/keyring default
>> > leveldb_log default
>> > lockdep 1 file
>> > log_file /home/sage/src/ceph6/build/out/$name.log file
>> > mon_osd_backfillfull_ratio 0.99 file
>> > mon_osd_full_ratio 0.99 file
>> > mon_osd_nearfull_ratio 0.99 file
>> > mon_pg_warn_min_per_osd 3 mon
>> > osd_check_max_object_name_len_on_startup 0 file
>> > osd_class_default_list * file
>> > osd_class_dir /home/sage/src/ceph6/build/lib file
>> > osd_class_load_list * file
>> > osd_copyfrom_max_chunk 524288 mon
>> > osd_data /home/sage/src/ceph6/build/dev/osd$id file
>> > osd_debug_misdirected_ops 1 mon
>> > osd_debug_op_order 1 mon
>> > osd_failsafe_full_ratio 0.99 file
>> > osd_journal /home/sage/src/ceph6/build/dev/osd$id/journal file
>> > osd_journal_size 100 file
>> > osd_objectstore bluestore override file
>> > osd_pool_default_min_size 1 mon
>> > osd_pool_default_size 1 mon
>> > osd_scrub_load_threshold 2000 mon
>> > pid_file /home/sage/src/ceph6/build/out/$name.pid file
>> > plugin_dir /home/sage/src/ceph6/build/lib file
>> > run_dir /home/sage/src/ceph6/build/out file
>> > gnit:build (wip-config) 11:29 AM $ bin/ceph config set osd/host:gnit debug_filestore 20
>> > gnit:build (wip-config) 11:32 AM $ bin/ceph config get osd.0
>> > WHO MASK OPTION VALUE
>> > global crush_chooseleaf_type 0
>> > osd host:gnit debug_filestore 20/20
>> > global host:gnit debug_monc 15/15
>> > osd.0 debug_osd 33/33
>> > global mon_pg_warn_min_per_osd 3
>> > osd osd_copyfrom_max_chunk 524288
>> > osd osd_debug_misdirected_ops true
>> > osd osd_debug_op_order true
>> > global osd_pool_default_min_size 1
>> > global osd_pool_default_size 1
>> > osd osd_scrub_load_threshold 2000
>> > gnit:build (wip-config) 11:32 AM $ bin/ceph config set osd/class:ssd debug_bluestore 0
>> > gnit:build (wip-config) 11:33 AM $ bin/ceph config set osd/class:hdd debug_bluestore 20
>> > gnit:build (wip-config) 11:33 AM $ bin/ceph config get osd.0
>> > WHO MASK OPTION VALUE
>> > global crush_chooseleaf_type 0
>> > osd class:ssd debug_bluestore 0/0
>> > osd host:gnit debug_filestore 20/20
>> > global host:gnit debug_monc 15/15
>> > osd.0 debug_osd 33/33
>> > global mon_pg_warn_min_per_osd 3
>> > osd osd_copyfrom_max_chunk 524288
>> > osd osd_debug_misdirected_ops true
>> > osd osd_debug_op_order true
>> > global osd_pool_default_min_size 1
>> > global osd_pool_default_size 1
>> > osd osd_scrub_load_threshold 2000
>> >
>> > (osd.0 is an ssd)
>> >
>> > Thoughts?
>> > sage
>> > --
>> > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>> > the body of a message to majordomo@vger.kernel.org
>> > More majordomo info at http://vger.kernel.org/majordomo-info.html
>>
>>
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: mon config update
2018-01-05 18:39 ` Gregory Farnum
@ 2018-01-05 21:00 ` Sage Weil
0 siblings, 0 replies; 10+ messages in thread
From: Sage Weil @ 2018-01-05 21:00 UTC (permalink / raw
To: Gregory Farnum; +Cc: ceph-devel
[-- Attachment #1: Type: TEXT/PLAIN, Size: 9130 bytes --]
On Fri, 5 Jan 2018, Gregory Farnum wrote:
> On Fri, Jan 5, 2018 at 10:24 AM, Sage Weil <sweil@redhat.com> wrote:
> > On Fri, 5 Jan 2018, Gregory Farnum wrote:
> >> On Fri, Jan 5, 2018 at 9:35 AM, Sage Weil <sweil@redhat.com> wrote:
> >> > My mon-based config branch is coming together. The last bit I did was the
> >> > CLI commands to set and show config. It didn't come out quite like I
> >> > expected it would. Here's what there is:
> >> >
> >> > config dump Show all configuration option(s)
> >> > config get <who> {<key>} Show configuration option(s) for an entity
> >> > config rm <who> <name> Clear a configuration option for one or more
> >> > entities
> >> > config set <who> <name> <value> Cet a configuration option for one or more
> >> > entities
> >> > config show <who> Show running configuration
> >> >
> >> > The show command is the oddball here: it reports what the *daemon* reports
> >> > as the running config, which includes other inputs (conf file, overrides,
> >> > etc.), while the 'get' and 'dump' are just what the mon stores. I wonder
> >> > if there is a better command than 'show'? Maybe 'running' or 'active' or
> >> > 'report' or something?
> >>
> >> I don't have a good suggestion for this name, but I was a little
> >> puzzled until I realized it seems to only show the options which are
> >> non-default? Given that defaults can change across software versions,
> >> I'm not sure that's a feasible long-term strategy. (In addition to
> >> dealing with whether we're comparing against the defaults on the OSD
> >> version or the monitor version, if I get a dump of options from
> >> somebody to do debugging with I'm not going to remember that a timeout
> >> changed between point releases and they're operating differently!)
> >>
> >> Although I definitely get the desire to make the config legible when
> >> dumped, I'm not sure there's much utility there as a default anyway —
> >> won't admins mostly be querying for a specific value or else running
> >> diffs between different dumps? Perhaps we can deal with the bad naming
> >> by having "ceph config show non-default <who>", "ceph config show
> >> current <who>", and whatever else we like.
> >
> > Yeah, defaults are excluded because there are 1000+ options and the output
> > becomes unusable. I'd prefer to make the less-standard command the one
> > that dumps everything (including defaults), like show-all, or similar.
> > Note though that currently the daemons aren't reporting their default
> > values to the mgr at all--this keeps the MMgrReport message smaller.
>
> Hmm. Like I said, this is going to kill somebody on debugging someday.
> I think we just need to eat it. (We could only send the config options
> when one changes, if we're worried about their size or parsing cost.)
I'll have the default values reported on mgr session open only; that
should be lightweight.
and 'ceph config show-all' or '... show-with-defaults'?
> >> (Maybe options for showing configs under USER, ADVANCED, and DEV?)
> >
> > Oh yeah, I forgot about these! I guess I should add columns for these for
> > both 'dump' and 'get'?
> >
> > Also, I guess the other main missing piece is 'ceph config help <option>',
> > which should present a friendly summary of the option, its description,
> > defaults, and so on?
> >
> >> Looking at the output, the WHO versus MASK bit is a little thorny. Can
> >> you show what it looks like when there are multiple masks on the same
> >> option?
> >
> > gnit:build (wip-config) 12:15 PM $ bin/ceph config set osd/class:ssd/host:gnit debug_bluestore 20
> > gnit:build (wip-config) 12:15 PM $ bin/ceph config get osd.0 debug_bluestore
> > WHO MASK OPTION VALUE
> > osd host:gnit/class:ssd debug_bluestore 20/20
> > gnit:build (wip-config) 12:15 PM $ bin/ceph config set osd.0/class:ssd/host:gnit debug_bluestore 21
> > gnit:build (wip-config) 12:15 PM $ bin/ceph config get osd.0 debug_bluestore
> > WHO MASK OPTION VALUE
> > osd.0 host:gnit/class:ssd debug_bluestore 21/21
>
> I was unclear. I mean, what's it look like when there's a setting for
> osd/class:ssd and a different setting for osd/class:hdd? ...which I
> now realize you did have, but I missed it and that's a good
> illustration for what I meant. See end of email.
>
> >> It looks like things are dumped in config declaration order, right?
> >> Given that, maybe OPTION should be the "top" (leftmost) column and
> >> then WHO and MASK can be filed underneath it, with nicely indented
> >> output to make it more like a tree.
> >> Or else make it more like a tree in the other direction, with either
> >> WHO or MASK as the root and with output order controlled that way. But
> >> that probably involved a lot more repetition of the output.
> >
> > They're sorted by section (global, type, daemon), then by option name.
> > Indenting is a great idea... I'd probably indent just the 'who' portion,
> > like so? Is that what you mean?
> >
> > WHO MASK OPTION VALUE
> > global crush_chooseleaf_type 0
> > global mon_pg_warn_min_per_osd 3
> > global osd_pool_default_min_size 1
> > global osd_pool_default_size 1
> > client admin_socket /tmp/ceph-asok.A5CIZs/$name.$pid.asok
> > client log_file /home/sage/src/ceph6/build/out/$name.$pid.log
> > client.rgw rgw_crypt_require_ssl false
> > client.rgw rgw_crypt_s3_kms_encryption_keys testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo=
> > client.rgw rgw_frontends civetweb port=8000
> > client.rgw rgw_lc_debug_interval 10
>
> So that's not what I meant, but I like it if we're going this way.
>
> > mds debug_mds 20/20
> > mds debug_mgrc 20/20
> > osd bluestore_block_create true
> > osd bluestore_block_db_create true
> > osd debug_bluestore 30/30
> > osd host:gnit/class:hdd debug_bluestore 20/20
> > osd host:gnit/class:ssd debug_bluestore 20/20
> > osd.0 host:gnit/class:ssd debug_bluestore 21/21
> >
> > I'm not sure sorting by the option name would work well here since you'd
> > have, say, all of the debug_* options grouped together but applying to
> > different things. The idea was to easily see what the osd configs are,
> > similar to what you get when looking at the old ini file. And then also
> > see any specializations per daemon below it.
>
>
> Here's (a bad manually-typed illustration of) what I was suggesting:
>
> OPTION WHO MASK
> VALUE
> log_file client
> /tmp/ceph-asok.A5CIZs/$name.$pid.asok
> mds
> /tmp/ceph-asok.A5CIZs/mds.$id.$pid.asok
>
> host:gnit
> /tmp/ceph-asok.special_gnit/mds.$id.$pid.asok
> bluestore_block_create osd
> true
> bluestore_block_db_create osd
> true
> debug_bluestore osd
> host:gnit/class:hdd 20/20
>
> host:gnit/class:ssd 20/20
> osd.0
> host:gnit/class:ssd 21/21
> debug_filestore osd
> 10/10
>
> So this way the options which have different settings can be visually
> identified; and the same for masks versus daemon-type. (Although I'm
> not sure which order we want those columns to go in.)
That was pretty mangled, but I think see what you're after: grouping by
config option, then enumerating all the various values for it.
I'm still partial to the group-by-section view. I think it will be more
helpful to easily grok what options apply to thing x (or things of type x)
than to see how a single option varies across things of different types
(e.g., who cares how osd's debug_ms compares to mds's debug_ms?). Any
other opinions?
If we really want both we could do 'ceph config dump-by-*' with the
different sorting...
sage
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: mon config update
2018-01-05 17:35 mon config update Sage Weil
2018-01-05 18:11 ` Gregory Farnum
@ 2018-01-08 9:02 ` Piotr Dałek
2018-01-08 13:12 ` Sage Weil
2018-01-11 15:10 ` Sage Weil
1 sibling, 2 replies; 10+ messages in thread
From: Piotr Dałek @ 2018-01-08 9:02 UTC (permalink / raw
To: Sage Weil, ceph-devel
On 18-01-05 06:35 PM, Sage Weil wrote:
> My mon-based config branch is coming together. The last bit I did was the
> CLI commands to set and show config. It didn't come out quite like I
> expected it would. Here's what there is:
>
> config dump Show all configuration option(s)
> config get <who> {<key>} Show configuration option(s) for an entity
> config rm <who> <name> Clear a configuration option for one or more
> entities
> config set <who> <name> <value> Cet a configuration option for one or more
> entities
> config show <who> Show running configuration
>
> The show command is the oddball here: it reports what the *daemon* reports
> as the running config, which includes other inputs (conf file, overrides,
> etc.), while the 'get' and 'dump' are just what the mon stores. I wonder
> if there is a better command than 'show'? Maybe 'running' or 'active' or
> 'report' or something?
Shouldn't "config get <key>" work like a subset of "config show" (output a
specified <key> from daemon's running config)? Or, have "config show" accept
specific key. Both are fine for with me.
> A sample from my vstart cluster (vstart is only putting some of its
> options in the mon so far, so this shows both conf and mon as sources):
>
> gnit:build (wip-config) 11:28 AM $ bin/ceph config dump
> WHO MASK OPTION VALUE
> global crush_chooseleaf_type 0
> global mon_pg_warn_min_per_osd 3
> global osd_pool_default_min_size 1
> global osd_pool_default_size 1
> mds mds_debug_auth_pins true
> mds mds_debug_frag true
> mds mds_debug_subtrees true
> mon mon_allow_pool_deletes true
> [..]
> gnit:build (wip-config) 11:33 AM $ bin/ceph config get osd.0
> WHO MASK OPTION VALUE
> global crush_chooseleaf_type 0
> osd class:ssd debug_bluestore 0/0
> osd host:gnit debug_filestore 20/20
> global host:gnit debug_monc 15/15
> osd.0 debug_osd 33/33
> global mon_pg_warn_min_per_osd 3
> osd osd_copyfrom_max_chunk 524288
> osd osd_debug_misdirected_ops true
> osd osd_debug_op_order true
> global osd_pool_default_min_size 1
> global osd_pool_default_size 1
> osd osd_scrub_load_threshold 2000
>
> (osd.0 is an ssd)
>
> Thoughts?
I assume that this will support json output.
Wouldn't mind having a "changeable in runtime" column in these outputs. I
know it's semi-reliable, but it's still better than experimenting or
searching for it in code (which may be difficult for non-devs).
--
Piotr Dałek
piotr.dalek@corp.ovh.com
https://www.ovh.com/us/
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: mon config update
2018-01-08 9:02 ` Piotr Dałek
@ 2018-01-08 13:12 ` Sage Weil
2018-01-08 13:26 ` Piotr Dałek
2018-01-11 15:10 ` Sage Weil
1 sibling, 1 reply; 10+ messages in thread
From: Sage Weil @ 2018-01-08 13:12 UTC (permalink / raw
To: Piotr Dałek; +Cc: ceph-devel
[-- Attachment #1: Type: TEXT/PLAIN, Size: 3490 bytes --]
On Mon, 8 Jan 2018, Piotr Dałek wrote:
> On 18-01-05 06:35 PM, Sage Weil wrote:
> > My mon-based config branch is coming together. The last bit I did was the
> > CLI commands to set and show config. It didn't come out quite like I
> > expected it would. Here's what there is:
> >
> > config dump Show all configuration
> > option(s)
> > config get <who> {<key>} Show configuration option(s)
> > for an entity
> > config rm <who> <name> Clear a configuration option
> > for one or more
> > entities
> > config set <who> <name> <value> Cet a configuration option
> > for one or more
> > entities
> > config show <who> Show running configuration
> >
> > The show command is the oddball here: it reports what the *daemon* reports
> > as the running config, which includes other inputs (conf file, overrides,
> > etc.), while the 'get' and 'dump' are just what the mon stores. I wonder
> > if there is a better command than 'show'? Maybe 'running' or 'active' or
> > 'report' or something?
>
> Shouldn't "config get <key>" work like a subset of "config show" (output a
> specified <key> from daemon's running config)? Or, have "config show" accept
> specific key. Both are fine for with me.
'config get <who> <key>' works, but doesn't show you the default if
the option isn't set (will fix that). 'config show <who>' doesn't take
a key name; can fix that too.
> > A sample from my vstart cluster (vstart is only putting some of its
> > options in the mon so far, so this shows both conf and mon as sources):
> >
> > gnit:build (wip-config) 11:28 AM $ bin/ceph config dump
> > WHO MASK OPTION VALUE
> > global crush_chooseleaf_type 0
> > global mon_pg_warn_min_per_osd 3
> > global osd_pool_default_min_size 1
> > global osd_pool_default_size 1
> > mds mds_debug_auth_pins true
> > mds mds_debug_frag true
> > mds mds_debug_subtrees true
> > mon mon_allow_pool_deletes true
> > [..]
> > gnit:build (wip-config) 11:33 AM $ bin/ceph config get osd.0
> > WHO MASK OPTION VALUE
> > global crush_chooseleaf_type 0
> > osd class:ssd debug_bluestore 0/0
> > osd host:gnit debug_filestore 20/20
> > global host:gnit debug_monc 15/15
> > osd.0 debug_osd 33/33
> > global mon_pg_warn_min_per_osd 3
> > osd osd_copyfrom_max_chunk 524288
> > osd osd_debug_misdirected_ops true
> > osd osd_debug_op_order true
> > global osd_pool_default_min_size 1
> > global osd_pool_default_size 1
> > osd osd_scrub_load_threshold 2000
> >
> > (osd.0 is an ssd)
> >
> > Thoughts?
>
> I assume that this will support json output.
> Wouldn't mind having a "changeable in runtime" column in these outputs. I know
> it's semi-reliable, but it's still better than experimenting or searching for
> it in code (which may be difficult for non-devs).
What about in the JSON output only? Concerned about spending horizontal
space on a binary flag...
sage
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: mon config update
2018-01-08 13:12 ` Sage Weil
@ 2018-01-08 13:26 ` Piotr Dałek
0 siblings, 0 replies; 10+ messages in thread
From: Piotr Dałek @ 2018-01-08 13:26 UTC (permalink / raw
To: Sage Weil; +Cc: ceph-devel
On 18-01-08 02:12 PM, Sage Weil wrote:
>>> A sample from my vstart cluster (vstart is only putting some of its
>>> options in the mon so far, so this shows both conf and mon as sources):
>>>
>>> gnit:build (wip-config) 11:28 AM $ bin/ceph config dump
>>> WHO MASK OPTION VALUE
>>> global crush_chooseleaf_type 0
>>> global mon_pg_warn_min_per_osd 3
>>> global osd_pool_default_min_size 1
>>> global osd_pool_default_size 1
>>> mds mds_debug_auth_pins true
>>> mds mds_debug_frag true
>>> mds mds_debug_subtrees true
>>> mon mon_allow_pool_deletes true
>>> [..]
>>> gnit:build (wip-config) 11:33 AM $ bin/ceph config get osd.0
>>> WHO MASK OPTION VALUE
>>> global crush_chooseleaf_type 0
>>> osd class:ssd debug_bluestore 0/0
>>> osd host:gnit debug_filestore 20/20
>>> global host:gnit debug_monc 15/15
>>> osd.0 debug_osd 33/33
>>> global mon_pg_warn_min_per_osd 3
>>> osd osd_copyfrom_max_chunk 524288
>>> osd osd_debug_misdirected_ops true
>>> osd osd_debug_op_order true
>>> global osd_pool_default_min_size 1
>>> global osd_pool_default_size 1
>>> osd osd_scrub_load_threshold 2000
>>>
>>> (osd.0 is an ssd)
>>>
>>> Thoughts?
>>
>> I assume that this will support json output.
>> Wouldn't mind having a "changeable in runtime" column in these outputs. I know
>> it's semi-reliable, but it's still better than experimenting or searching for
>> it in code (which may be difficult for non-devs).
>
> What about in the JSON output only? Concerned about spending horizontal
> space on a binary flag...
A single asterisk character before/after option name - without additional
column - would do good enough. I'm quite often asked whether some particular
option can be changed at runtime or not, so that would make everyone's life
easier. Sure, this doesn't cover the cases of runtime-changeable options
that are not explicitly observed (confusing people in the process), but
that's another story.
--
Piotr Dałek
piotr.dalek@corp.ovh.com
https://www.ovh.com/us/
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: mon config update
2018-01-08 9:02 ` Piotr Dałek
2018-01-08 13:12 ` Sage Weil
@ 2018-01-11 15:10 ` Sage Weil
2018-01-11 20:18 ` Patrick Donnelly
1 sibling, 1 reply; 10+ messages in thread
From: Sage Weil @ 2018-01-11 15:10 UTC (permalink / raw
To: Piotr Dałek; +Cc: ceph-devel
[-- Attachment #1: Type: TEXT/PLAIN, Size: 2753 bytes --]
On Mon, 8 Jan 2018, Piotr Dałek wrote:
> Wouldn't mind having a "changeable in runtime" column in these outputs. I know
> it's semi-reliable, but it's still better than experimenting or searching for
> it in code (which may be difficult for non-devs).
Several parts of this.
1. 'config help'
$ bin/ceph config help debug_osd
debug_osd - Debug level for osd
(std::string, advanced)
Default: 1/5
Can update at runtime: true
(also in the JSON dump).
2. 'config dump' and 'config get' have a new "RW" column (suggestions for
a better short column name are welcome!):
$ bin/ceph config get osd.0
WHO MASK LEVEL OPTION VALUE RW
osd advanced debug_bdev 20/20 Y
osd advanced debug_bluefs 20/20 Y
osd advanced debug_bluestore 30/30 Y
osd advanced debug_filestore 20/20 Y
osd advanced debug_journal 20/20 Y
osd advanced debug_ms 1/1 Y
global advanced mon_pg_warn_min_per_osd 3 Y
osd advanced osd_copyfrom_max_chunk 524288 Y
global advanced osd_crush_chooseleaf_type 0 Y
osd developer osd_debug_misdirected_ops true Y
osd developer osd_debug_op_order true Y
osd.0 advanced osd_objectstore foo
3. We obviously need to improve the is_safe() implementation to make this
more accurate. My current plan is add annotations for if the option can
update at runtime, only takes effect at startup, at daemon creation time,
or cluster creation time. In most cases the guess is right, so hopefully
not too many annotations will be needed.
4. The 'config show' now indicates if a setting the mon tried to set
failed to take effect:
$ bin/ceph config show osd.0
NAME VALUE SOURCE OVERRIDES IGNORES
...
osd_journal_size 100 file
osd_objectstore bluestore file mon
osd_pool_default_min_size 1 mon
osd_pool_default_size 1 mon
...
The IGNORES column is meant to make sense next to SOURCE and OVERRIDES.
For example, mon_data source is mon, overrides nothing, but ignores 'mon'.
In reality, the only value IGNORES will currently ever have is 'mon',
since trying to do e.g. 'ceph daemon ... config set ...' will error out
without setting anything so you never get into a conflicting state. In
principle we could eventually show soenthing else here if there is a
setting that is forced based on on-disk state (e.g., osd_objectstore) that
might conflict with a setting that only works at daemon creation time.
In reality we just ignore setting in that case; I'm not sure if it would
make sense to show it here (or would it?).
sage
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: mon config update
2018-01-11 15:10 ` Sage Weil
@ 2018-01-11 20:18 ` Patrick Donnelly
0 siblings, 0 replies; 10+ messages in thread
From: Patrick Donnelly @ 2018-01-11 20:18 UTC (permalink / raw
To: Sage Weil; +Cc: Piotr Dałek, Ceph Development
On Thu, Jan 11, 2018 at 7:10 AM, Sage Weil <sage@newdream.net> wrote:
> 2. 'config dump' and 'config get' have a new "RW" column (suggestions for
> a better short column name are welcome!):
"RO" (read-only) is a bit more common I think and clear in intent.
(i.e. RW false mean no reads?)
--
Patrick Donnelly
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2018-01-11 20:18 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2018-01-05 17:35 mon config update Sage Weil
2018-01-05 18:11 ` Gregory Farnum
2018-01-05 18:24 ` Sage Weil
2018-01-05 18:39 ` Gregory Farnum
2018-01-05 21:00 ` Sage Weil
2018-01-08 9:02 ` Piotr Dałek
2018-01-08 13:12 ` Sage Weil
2018-01-08 13:26 ` Piotr Dałek
2018-01-11 15:10 ` Sage Weil
2018-01-11 20:18 ` Patrick Donnelly
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.