From: Sage Weil <sweil@redhat.com>
To: Wei-Chung Cheng <freeze.vicente.cheng@gmail.com>
Cc: David Zafman <dzafman@redhat.com>,
Loic Dachary <loic@dachary.org>,
Ceph Development <ceph-devel@vger.kernel.org>
Subject: Re: new OSD re-using old OSD id fails to boot
Date: Wed, 9 Dec 2015 06:00:06 -0800 (PST) [thread overview]
Message-ID: <alpine.DEB.2.00.1512090555030.12816@cobra.newdream.net> (raw)
In-Reply-To: <CABF_e-FC7TGFw9=_zEB2-GX=3EOcoX-oc6=L=NPbuipBRo+atg@mail.gmail.com>
On Wed, 9 Dec 2015, Wei-Chung Cheng wrote:
> Hi Loic,
>
> I try to reproduce this problem on my CentOS7.
> I can not do the same issue.
> This is my version:
> ceph version 10.0.0-928-g8eb0ed1 (8eb0ed1dcda9ee6180a06ee6a4415b112090c534)
> Would you describe more detail?
>
>
> Hi David, Sage,
>
> In most of time, when we found the osd failure, the OSD is already in
> `out` state.
> It could not avoid the redundant data movement unless we could set the
> osd noout when failure.
> Is it right? (Means if OSD go into `out` state, it will make some
> redundant data movement)
>
> Could we try the traditional spare behavior? (Set some disks backup
> and auto replace the broken device?)
>
> That can replace the failure osd before it go into the `out` state.
> Or we could always set the osd noout?
I don't think there is a problem with 'out' if the osd id is reused and
the crush position remains the same. And I expect usually the OSD will be
replaced by a disk with a similar size. If the replacement is smaller (or
0--removed entirely) then you get double-movement, but if it's the same or
larger I think it's fine.
The sequence would be something like
up + in
down + in
5-10 minutes go by
down + out (marked out by monitor)
new replicas uniformly distributed across cluster
days go by
disk removed
new disk inserted
ceph-disk recreate ... recreates osd dir w/ the same id, new uuid
on startup, osd adjusts crush weight (maybe.. usually by a smallish amount)
up + in
replicas migrate back to new device
sage
next prev parent reply other threads:[~2015-12-09 14:00 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-12-05 16:49 new OSD re-using old OSD id fails to boot Loic Dachary
2015-12-09 1:13 ` David Zafman
2015-12-09 2:50 ` Sage Weil
2015-12-09 10:39 ` Wei-Chung Cheng
2015-12-09 13:08 ` Loic Dachary
2015-12-09 14:00 ` Sage Weil [this message]
2015-12-09 19:55 ` David Zafman
2015-12-09 20:08 ` Sage Weil
2015-12-09 13:06 ` Loic Dachary
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=alpine.DEB.2.00.1512090555030.12816@cobra.newdream.net \
--to=sweil@redhat.com \
--cc=ceph-devel@vger.kernel.org \
--cc=dzafman@redhat.com \
--cc=freeze.vicente.cheng@gmail.com \
--cc=loic@dachary.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.