From: Daniel Vetter <daniel.vetter@ffwll.ch>
To: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Cc: ksummit <ksummit-discuss@lists.linuxfoundation.org>
Subject: Re: [Ksummit-discuss] [MAINTAINER SUMMIT] community management/subsystem governance
Date: Mon, 10 Sep 2018 22:55:22 +0200 [thread overview]
Message-ID: <CAKMK7uFAXoVG+nBX=Om9A7R9xPYa-hd0JrRX1RNKSNhxnL_VsA@mail.gmail.com> (raw)
In-Reply-To: <8412864.7ztUKcXNNC@avalon>
On Mon, Sep 10, 2018 at 10:32 PM, Laurent Pinchart
<laurent.pinchart@ideasonboard.com> wrote:
> On Monday, 10 September 2018 18:56:18 EEST Daniel Vetter wrote:
>> On Mon, Sep 10, 2018 at 4:53 PM, Linus Torvalds wrote:
>> > On Sun, Sep 9, 2018 at 10:59 PM Daniel Vetter wrote:
>> >> - Assuming it gets accepted I think my LPC talk on "migrating to
>> >> gitlab" will raise some questions, and probably overflow into hallway
>> >> track or a BOF session.
>>
>> First a bit more context: This is a different talk than the other
>> "stuff drm people are doing" talks I've done in the past: We haven't
>> done anything yet. We haven't even really started planning the
>> migration yet :-)
>>
>> The idea here was that for once I don't do a talk when it's all done,
>> but way ahead of that when we're still figuring things out. The
>> underlying reason for the migration is that fd.o admins want to shut
>> down their home-grown collage of services and replace it with
>> something that's much easier to maintain, provides solid 2FA and
>> allows them to offload a lot of the admin work (handing out access
>> rights and stuff) to all the projects their hosting. Plus stop the
>> shell access services, if that's ever possible.
>>
>> So right now this is only about moving the git server from one place
>> to the other. And we haven't even done the full plan for that
>> migration (it's a pile of repos and stuff). So really very early. But
>> now that we do have this modern thing, a lot of people are looking
>> into all the shiny features it offers, and trying to figure out
>> whether they're useful for us or not. And that's what I want to go
>> through in the talk - the title is a bit on the clickbait side :-)
>>
>> > I've not used gitlab personally, but I *have* used github for a much
>> > smaller project.
>> >
>> > I have to say, the random and relaxed model is enjoyable. I can see
>> > how somebody coming from that, then finds the strict kernel rules (and
>> > _different_ rules for different parts) off-putting and confusing.
>> >
>> > At the same time, I have to say that people need to keep in mind that
>> > the kernel is *different*. We're not a small project with five
>> > developers that isn't all that critical. Some of our off-putting
>> > development models are there for a very very good reason. I think a
>> > lot of people who find the kernel unfriendly just don't appreciate
>> > that part.
>> >
>> > The kernel used to be pretty free-wheeling too. 20+ years ago.
>>
>> What I definitely don't want is a free-wheeling thing a la your
>> standard github repo. There's also the issue that github massively
>> optimizes for small projects in their entire setup. I've done a blog
>> post about that a while ago and chatted with a bunch of githubbers,
>> they're really not interested in big-project workflows that much.
>>
>> gitlab otoh is very interested in that, and they picked up some of the
>> concepts around how we use topic branches and pull requests in the
>> kernel. It's by far not all implemented yet, but a lot more promising
>> than anything else I've looked at. And we can always just keep using
>> our existing tooling ofc.
>>
>> Aside from this there's a bunch of other reasons fd.o admins picked
>> gitlab (against other contenders they checked out), but this is one of
>> the big ones. At least from the kernel's pov. For anyone bored enough,
>> here's the full details of the why from fd.o admins:
>>
>> https://www.fooishbar.org/blog/gitlab-fdo-introduction/
>>
>> > And I still hate how github ends up making it really really easy to
>> > make horribly bad commit messages, and it encourages a "just rebase on
>> > top of the integration branch" model, and I do not believe that it
>> > would ever work for the kernel at large. Too much room for chaos.
>> >
>> > BUT.
>> >
>> > I do think it's still instructive to look at how those "fun small
>> > projects" work. Having the whole web interface and a more relaxed
>> > setup is a good thing. And it's probably *better* than the strict
>> > rules when you don't really need those strict rules.
>> >
>> > So I do believe that it could work for a subsystem. Because "too much
>> > room for chaos" ends up being nice when you don't want to worry about
>> > the proper channels etc.
>> >
>> > For example, we've had the "trivial tree", which tends to be a really
>> > thankless project, that might well be managed way more easily by just
>> > having a random tree that lots of people can commit to, and we could
>> > even encourage the github (gitlab?) model of random non-kernel people
>> > just sending their random trees to it, and have then the group of
>> > committers be able to merge the changes (and at least on github, the
>> > default merge is just a fast-forward, so it actually acts more like a
>> > patch queue than a git tree).
>>
>> Yeah, the trivial/"fun first contributions" process is a lot more
>> polished on many of these projects, and there's a bunch such things we
>> kinda want to look into longer-term. And this really is all long-term,
>> because just the initial git server migration most likely won't even
>> start until next year.
>>
>> A much bigger thing for us is CI integration. patchwork + patch bombs
>> + lots of tears and gin do kinda work, but it's super painful and a
>> real chaos no one has any overview over. gitlab has really neat
>> integrated CI for anything that you can run on containers/virtual
>> machines (already played with it, pretty dope, and I have 0 clue about
>> this docker/container stuff). And also seems to have good support to
>> tie all kinds of exterrnal CI validation into pull requests (well
>> gitlab calls them merge requests, it's the same).
>>
>> One thing we're at least planning to experiment with is to take the
>> driver pull requests from drm, auto-convert them into gitlab merge
>> requests (or just ask people to directly do that), and then let to
>> bots go wild. Would give us a nice single point where you can check
>> all the information from all the different checks and vendor CI and
>> everything. Not even that much with the goal to do the merge using the
>> web ui, just to integrate all the testing and validation in one spot.
>>
>> > And the reason I mention the trivial tree is not because the trivial
>> > tree itself is all that interesting or because I'd like to belittle
>> > that model ("that will only work for trivial unimportant stuff"), but
>> > because it might be a good area to experiment in, and a way to get
>> > people used to the flow.
>> >
>> > Because if somebody is willing to every once in a while look at
>> > trivial tree pull requests and merge them to the trivial tree, maybe
>> > that person will start using the same flow for their "real" work.
>> >
>> > And I do think that "patches by email" doesn't scale. I've been there,
>> > done that, and I got the T-shirt.
>> >
>> > I used tools that some people absolutely hated to get out of that
>> > rat-hole. When that failed, I had to write my own.
>> >
>> > So I very much do think that email doesn't really work at scale.
>> >
>> > But I know the kernel people who still do real development (as opposed
>> > to me) work that way.
>> >
>> > So let me suggest a topic for the maintainer summit:
>> > "Live without email - possible?"
>> >
>> > just to get that ball rolling
>>
>> I'd say "nope", even with the s/live/patches/. Definitely not for
>> anything spanning subsystems. Even within gitlabd the cross-tree
>> support is not (yet) there. And we definitely need to have a solid
>> gitlab/mailing list gateway, which is somewhere on the huge list of
>> things fd.o admins need to do. It exists, but SMTP is pain, and fd.o
>> is volunteer run, so all takes time. And then the gateway might be too
>> much garbage to be useful.
>
> I believe we need a common communication system that is supported by all
> subsystems. Today that system is email.
>
> If we don't want to challenge that, we'll need first-class compatibility
> between whatever new tool we introduce and email, in both direction. All
> discussions, including patch review, happening through other means (web UI for
> instance) will need to be pushed to emails, and email replies will need to be
> integrated back into the system, in such a way that will not completely mess
> up the formatting and communication flow. It has to be transparent enough.
>
> If we want to challenge that, we'll have to agree about another communication
> system for the whole kernel. While I don't think that's realistic, I'm not
> opposed to discussing it. However, I expect many developers to come up with
> lists of requirements, and the result of merging all those requirements
> together to be exactly email and nothing else :-)
>
> On my side, there are three very important features that a communication
> system should have:
>
> - It should be push-based. I don't want to have to log in and pull information
> from web UIs.
>
> - It should be customizable, offering an easy way to script review, and patch
> and pull request handling.
>
> - It should offer an offline mode. Lots of us travel a lot or generally have
> to work without an internet connection more often than we would like, so any
> solution that requires being online won't work.
>
> Emails provide all that, there may be good options I just fail to think about
> right now.
Thanks for your list, "figuring out what we need from email to
evaluate the usefulness of the gitlab gateway" was somewhere on my
todo :-)
More seriously: We're _really_ not far beyond the "hey, what about
gitlab" stage of this entire discussions. As mentioned, I have not
much answers, much less a plan, right now. The mail goal here is just
to figure out the right questions (and then hopefully have some useful
progress to report at LPC already).
>> My goal at least with this is much more in figuring out new workflows,
>> and running a pile of experiments. As mentioned, we don't yet even
>> have a plan for all this, the goal here is to spark some discussions
>> and figure out whether maybe others want to come along for the ride.
>
> I recently had to deal with a new bugs and tasks management tool that was
> developed in-house by a customer of mine. The tool was presented to us in its
> first RFC version (and in source code form, which is very nice), with very few
> implemented features, and without telling us what process it was supposed to
> support. When I inquired I got told that the process hasn't been taken into
> consideration to develop the tool, and that it would just come into existence
> as a result of the tool. Needless to say, it was hard for me to review the
> code, without knowing what it was supposed to do.
>
> I don't claim we're going to that extreme, but I believe it would be useful to
> detail the problems we have in order to find solutions, instead of proposing
> solutions and then trying to retrofit them to problems.
We're definitely starting with some pain points, and don't do this
gitlab thing as an a solution looking desperately for a problem. With
my intel hat on there's a few main ones:
- CI integration with patchwork is a pain, for developers, maintainers
and CI people all alike. I plan to do some slides on what exactly
sucks and what gitlab is doing better, but I'm just not quite there
yet. As mentioned somewhere else, ideally we'd have the full demo
ready for LPC using the userspace igt gpu tests repo.
- First contributor flow can be (if done correctly) so much more fun
than mailing lists. There's been a bit of chatting on this already in
this thread.
- Tracking patch series as they evolve from RFC to final reviewed
version. Patchwork, even with all the extensions we have since years
in the fd.o one, just can't cope. This might be an artifact of our
group maintainership + committer model, I think for single maintainer
patchwork does work a bit better. This is the "patches on mailing
lists don't scale, I have the T-shirt" problem Linus already
mentioned. Our PMs/managers also don't like that they can't keep track
of what the fairly big drm/i915 team is doing :-)
Aside: If you want to be involved in the nitty gritty details (aka
lots of boring fd.o admin discussions), here's the overall migration
tracking task: https://gitlab.freedesktop.org/freedesktop/freedesktop/issues/69
>> Whether anything of this will make it into the production drm
>> processes, no idea. "maybe" is the best answer at this time I think.
>>
>> And yes I hope that by LPC I'll have a bit more solid understanding of
>> how this gitlab thing works and what we could try out, so this isn't
>> as much high-level gossipping as it is right now :-)
Cheers, Daniel
--
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch
next prev parent reply other threads:[~2018-09-10 20:55 UTC|newest]
Thread overview: 162+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-09-10 8:59 [Ksummit-discuss] [MAINTAINER SUMMIT] community management/subsystem governance Daniel Vetter
2018-09-10 14:53 ` Linus Torvalds
2018-09-10 15:08 ` James Bottomley
2018-09-10 15:10 ` Linus Torvalds
2018-09-10 15:38 ` Sasha Levin
2018-09-10 15:47 ` James Bottomley
2018-09-10 15:55 ` Sasha Levin
2018-09-10 16:13 ` James Bottomley
2018-09-10 16:24 ` Sasha Levin
2018-09-10 17:10 ` James Bottomley
2018-09-10 15:47 ` Konstantin Ryabitsev
2018-09-10 15:56 ` Sasha Levin
2018-09-10 16:02 ` Konstantin Ryabitsev
2018-09-10 16:07 ` Daniel Vetter
2018-09-10 16:18 ` Konstantin Ryabitsev
2018-09-10 16:23 ` Daniel Vetter
2018-09-10 16:41 ` Konstantin Ryabitsev
2018-09-10 17:06 ` Daniel Vetter
2018-09-10 19:48 ` Laurent Pinchart
2018-09-10 20:50 ` Daniel Vetter
2018-09-10 15:49 ` Mark Brown
2018-09-10 16:33 ` Olof Johansson
2018-09-10 19:59 ` Laurent Pinchart
2018-09-10 21:30 ` Josh Triplett
2018-09-10 23:00 ` Laurent Pinchart
2018-09-10 23:16 ` Daniel Vetter
2018-09-11 1:14 ` Josh Triplett
2018-09-10 15:13 ` Jiri Kosina
2018-09-10 15:20 ` James Bottomley
2018-09-10 15:31 ` Sasha Levin
2018-09-10 20:15 ` Laurent Pinchart
2018-09-10 21:09 ` Sean Paul
2018-09-10 21:38 ` Laurent Pinchart
2018-09-11 10:06 ` Leon Romanovsky
2018-09-11 8:44 ` Jani Nikula
2018-09-11 9:08 ` Geert Uytterhoeven
2018-09-11 10:01 ` Daniel Vetter
2018-09-11 10:09 ` Geert Uytterhoeven
2018-09-11 10:17 ` Daniel Vetter
2018-09-11 10:30 ` Geert Uytterhoeven
2018-09-11 8:41 ` Jani Nikula
2018-09-10 15:31 ` Daniel Vetter
2018-09-10 16:39 ` Olof Johansson
2018-09-10 17:10 ` Daniel Vetter
2018-09-12 19:02 ` Darren Hart
2018-09-12 18:59 ` Darren Hart
2018-09-12 20:05 ` Daniel Vetter
2018-09-12 20:58 ` Darren Hart
2018-09-13 11:27 ` Mark Brown
2018-09-13 11:41 ` Daniel Vetter
2018-09-13 17:08 ` Darren Hart
2018-09-13 2:56 ` Theodore Y. Ts'o
2018-09-13 5:17 ` Daniel Vetter
2018-09-10 15:56 ` Daniel Vetter
2018-09-10 20:32 ` Laurent Pinchart
2018-09-10 20:55 ` Daniel Vetter [this message]
2018-09-10 21:33 ` Laurent Pinchart
2018-09-10 22:44 ` Daniel Vetter
2018-09-11 12:44 ` Alexandre Belloni
2018-09-11 14:35 ` Mark Brown
2018-09-11 15:17 ` Alexandre Belloni
2018-09-11 15:02 ` Daniel Vetter
2018-09-11 22:00 ` Alexandre Belloni
2018-09-11 22:17 ` Guenter Roeck
2018-09-12 8:42 ` Jani Nikula
2018-09-12 18:45 ` Alexandre Belloni
2018-09-12 19:52 ` Dave Airlie
2018-09-12 22:25 ` Daniel Vetter
2018-09-12 9:14 ` Linus Walleij
2018-09-12 18:23 ` Alexandre Belloni
2018-09-12 18:44 ` Thomas Gleixner
2018-09-13 12:08 ` Maxime Ripard
2018-09-13 12:57 ` Alexandre Belloni
2018-09-13 13:18 ` Maxime Ripard
2018-09-13 14:25 ` Jani Nikula
2018-09-13 20:05 ` Thomas Gleixner
2018-09-13 23:02 ` Rodrigo Vivi
2018-09-14 6:47 ` Rafael J. Wysocki
2018-09-14 6:39 ` Dave Airlie
2018-09-14 14:15 ` Thomas Gleixner
2018-09-17 7:40 ` Daniel Vetter
2018-09-14 7:08 ` Linus Walleij
2018-09-14 7:39 ` Geert Uytterhoeven
2018-09-14 8:08 ` Linus Walleij
2018-09-12 21:21 ` Linus Walleij
2018-09-21 16:05 ` Joe Perches
2018-09-12 22:44 ` Laurent Pinchart
2018-09-10 22:56 ` Laurent Pinchart
2018-09-10 21:11 ` Theodore Y. Ts'o
2018-09-10 23:05 ` Laurent Pinchart
2018-09-17 11:43 ` Mauro Carvalho Chehab
2018-09-17 12:03 ` Geert Uytterhoeven
2018-09-17 13:04 ` Mauro Carvalho Chehab
2018-09-17 13:10 ` Julia Lawall
2018-09-17 13:29 ` Christoph Hellwig
2018-09-17 13:48 ` Laurent Pinchart
2018-09-17 13:58 ` Mauro Carvalho Chehab
2018-09-17 14:18 ` Christoph Hellwig
2018-09-17 14:50 ` Geert Uytterhoeven
2018-09-17 15:21 ` Mauro Carvalho Chehab
2018-09-17 14:18 ` Laurent Pinchart
2018-09-17 16:50 ` Joe Perches
2018-09-17 14:14 ` Laurent Pinchart
2018-09-17 14:59 ` Mauro Carvalho Chehab
2018-09-17 22:39 ` Dave Airlie
2018-09-17 23:04 ` James Bottomley
2018-09-18 8:00 ` Daniel Vetter
2018-09-18 11:16 ` James Bottomley
2018-09-18 15:26 ` Randy Dunlap
2018-09-18 16:47 ` Tim.Bird
2018-09-18 16:59 ` Konstantin Ryabitsev
2018-09-18 17:08 ` Tim.Bird
2018-09-18 17:12 ` Tim.Bird
2018-09-18 17:31 ` Konstantin Ryabitsev
2018-09-18 17:42 ` Tim.Bird
2018-09-18 17:55 ` Konstantin Ryabitsev
2018-09-18 18:58 ` Tim.Bird
2018-09-18 19:24 ` Konstantin Ryabitsev
2018-09-18 17:47 ` Geert Uytterhoeven
2018-09-18 17:49 ` Greg KH
2018-09-18 18:03 ` Konstantin Ryabitsev
2018-09-18 22:46 ` Alexandre Belloni
2018-09-18 18:22 ` Dmitry Torokhov
2018-09-18 19:16 ` Theodore Y. Ts'o
2018-09-18 18:56 ` Sasha Levin
2018-09-18 23:05 ` Laurent Pinchart
2018-09-18 7:37 ` Nicolas Ferre
2018-09-18 7:47 ` Geert Uytterhoeven
2018-09-18 10:38 ` Laurent Pinchart
2018-09-18 16:02 ` Mark Brown
2018-09-18 16:32 ` Luck, Tony
2018-09-18 16:35 ` Dmitry Torokhov
2018-09-18 17:18 ` Linus Torvalds
2018-09-18 17:28 ` Sean Paul
2018-09-18 17:37 ` Tim.Bird
2018-09-21 16:46 ` Olof Johansson
2018-09-21 17:08 ` Mauro Carvalho Chehab
2018-09-21 17:16 ` Olof Johansson
2018-09-18 17:21 ` Mark Brown
2018-09-18 21:01 ` Steven Rostedt
2018-09-18 23:16 ` Laurent Pinchart
2018-09-18 23:54 ` Mark Brown
2018-09-19 5:27 ` Christoph Hellwig
2018-09-19 9:46 ` James Bottomley
2018-09-18 17:10 ` Tim.Bird
2018-09-18 20:48 ` Takashi Iwai
2018-09-18 16:50 ` David Woodhouse
2018-09-18 17:24 ` Mark Brown
2018-09-18 19:22 ` David Woodhouse
2018-09-18 19:30 ` Sasha Levin
2018-09-18 19:38 ` Josh Triplett
2018-09-18 19:48 ` David Woodhouse
2018-09-18 8:24 ` Eric W. Biederman
2018-09-17 13:12 ` Christoph Hellwig
2018-09-17 14:14 ` Mauro Carvalho Chehab
2018-09-17 21:59 ` Rafael J. Wysocki
2018-09-17 22:17 ` Rafael J. Wysocki
2018-09-10 21:19 ` Konstantin Ryabitsev
2018-09-11 8:33 ` Rafael J. Wysocki
2018-09-10 16:29 ` [Ksummit-discuss] Fwd: " Daniel Vetter
2018-09-11 15:35 ` [Ksummit-discuss] " Jiri Kosina
2018-09-17 11:11 ` [Ksummit-discuss] [MAINTAINER SUMMIT] Live without email - possible? - Was: " Mauro Carvalho Chehab
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAKMK7uFAXoVG+nBX=Om9A7R9xPYa-hd0JrRX1RNKSNhxnL_VsA@mail.gmail.com' \
--to=daniel.vetter@ffwll.ch \
--cc=ksummit-discuss@lists.linuxfoundation.org \
--cc=laurent.pinchart@ideasonboard.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).