From: Phillip Wood <phillip.wood123@gmail.com>
To: Calvin Wan <calvinwan@google.com>, git@vger.kernel.org
Cc: gitster@pobox.com, jonathantanmy@google.com,
philipoakley@iee.email, johncai86@gmail.com
Subject: Re: [PATCH v5 5/6] transport: add client support for object-info
Date: Mon, 1 Aug 2022 14:30:00 +0100 [thread overview]
Message-ID: <b33ec478-8858-faa7-2678-f02559426539@gmail.com> (raw)
In-Reply-To: <20220728230210.2952731-6-calvinwan@google.com>
Hi Calvin
On 29/07/2022 00:02, Calvin Wan wrote:
> Sometimes it is useful to get information about an object without having
> to download it completely. The server logic has already been implemented
> as “a2ba162cda (object-info: support for retrieving object info,
> 2021-04-20)”. This patch adds client functions to communicate with the
> server.
>
> The client currently supports requesting a list of object ids with
> features 'size' and 'type' from a v2 server. If a server does not
> advertise either of the requested features, then the client falls back
> to making the request through 'fetch'.
I'm confused by the 'type' support, I might have missed something as I'm
not familiar with this code but I couldn't see where we parse the type
returned by the server.
> + for (i = 0; i < args.object_info_options->nr; i++) {
> + if (packet_reader_read(&reader) != PACKET_READ_NORMAL) {
> + check_stateless_delimiter(transport->stateless_rpc, &reader, "stateless delimiter expected");
This is one of a number of lines in this series that are way over the 80
column limit.
> + return -1;
> + }
> + if (unsorted_string_list_has_string(args.object_info_options, reader.line)) {
> + if (!strcmp(reader.line, "size"))
> + size_index = i;
Should we be checking for "type" as well? Also does protocol-v2.txt need
updating as it only mentions "size" as an attribute.
> + continue;
> + }
> + return -1;
> + }
> +
> + i = 0;
> + while (packet_reader_read(&reader) == PACKET_READ_NORMAL && i < args.oids->nr) {
> + struct string_list object_info_values = STRING_LIST_INIT_DUP;
> +
> + string_list_split(&object_info_values, reader.line, ' ', -1);
> + if (0 <= size_index) {
To avoid a possible out-of-bounds access we need to check that
size_index + 1 < object_info_value.nr in case the server response is
malformed
> + if (!strcmp(object_info_values.items[1 + size_index].string, ""))
> + die("object-info: not our ref %s",
I'm a bit confused by this message is it trying to say "object %s is
missing on the server"?
> + object_info_values.items[0].string);
> + *(*object_info_data)[i].sizep = strtoul(object_info_values.items[1 + size_index].string, NULL, 10);
As Junio pointed out in his comments in v4 there is no error checking
here - we should check the server has actually returned a number. Note
that strtoul() will happily parse negative numbers so we probably want
to do something like
const char *endp
errno = 0
if (!isdigit(*object_info_values.items[1 + size_index].string))
die("...")
*(*object_info_data)[i].sizep = strtoul(object_info_values.items[1 +
size_index].string, &endp, 10);
if (errno || *endp)
die("...")
Should be we checking the object id matches what we asked for? (I'm not
sure if protocol-v2.txt mentions the order in which the objects are
returned)
Should we be parsing the object type here as well?
> @@ -392,8 +468,25 @@ static int fetch_refs_via_pack(struct transport *transport,
> args.server_options = transport->server_options;
> args.negotiation_tips = data->options.negotiation_tips;
> args.reject_shallow_remote = transport->smart_options->reject_shallow;
> -
> - if (!data->got_remote_heads) {
> + args.object_info = transport->smart_options->object_info;
> +
> + if (transport->smart_options && transport->smart_options->object_info) {
> + struct ref *ref = object_info_refs;
> +
> + if (!fetch_object_info(transport, data->options.object_info_data))
> + goto cleanup;
> + args.object_info_data = data->options.object_info_data;
> + args.quiet = 1;
> + args.no_progress = 1;
> + for (i = 0; i < transport->smart_options->object_info_oids->nr; i++) {
> + struct ref *temp_ref = xcalloc(1, sizeof (struct ref));
Using CALLOC_ARRAY() or p = xcalloc(1, sizeof(*p)) would be safer here
(and everywhere else where you have used xcalloc()) as it ensures we
allocate the correct size.
> diff --git a/transport.h b/transport.h
> index b5bf7b3e70..5512fdb140 100644
> --- a/transport.h
> +++ b/transport.h
> @@ -31,6 +32,12 @@ struct git_transport_options {
> */
> unsigned connectivity_checked:1;
>
> + /*
> + * Transport will attempt to pull only object-info. Fallbacks
> + * to pulling entire object if object-info is not supported
> + */
Is it definitely true that we fallback to pulling the entire object? -
there is at least one place above where we do
> + if (transport->smart_options->object_info) {
> + die(_("--object-info requires protocol v2"));
> + }
>
Best Wishes
Phillip
next prev parent reply other threads:[~2022-08-01 13:30 UTC|newest]
Thread overview: 103+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-02-08 23:19 [PATCH] fetch —object-info-format: client option for object-info Calvin Wan
2022-02-08 23:56 ` [PATCH v2] " Calvin Wan
2022-02-09 12:48 ` Philip Oakley
2022-02-10 22:32 ` Calvin Wan
2022-02-09 20:41 ` [PATCH v2] fetch object-info-format: " Jonathan Tan
2022-02-10 22:58 ` Calvin Wan
2022-03-28 19:11 ` [PATCH v3 0/3] object-info: add option for retrieving object info Calvin Wan
2022-03-28 19:11 ` [PATCH v3 1/3] fetch-pack: refactor packet writing and fetch options Calvin Wan
2022-03-29 22:54 ` Junio C Hamano
2022-03-29 23:01 ` Taylor Blau
2022-03-30 21:55 ` Jonathan Tan
2022-03-28 19:11 ` [PATCH v3 2/3] transfer.advertiseObjectInfo: add object-info config Calvin Wan
2022-03-29 22:34 ` Junio C Hamano
2022-03-29 22:48 ` Calvin Wan
2022-03-29 23:09 ` Taylor Blau
2022-03-28 19:11 ` [PATCH v3 3/3] object-info: add option for retrieving object info Calvin Wan
2022-03-29 19:57 ` Junio C Hamano
2022-03-29 22:54 ` Junio C Hamano
2022-03-30 0:49 ` Junio C Hamano
2022-03-30 22:31 ` Calvin Wan
2022-03-30 22:43 ` Jonathan Tan
2022-03-30 23:42 ` Junio C Hamano
2022-03-29 23:19 ` Taylor Blau
2022-03-30 22:47 ` Calvin Wan
2022-03-30 22:06 ` John Cai
2022-03-31 19:56 ` Calvin Wan
2022-04-01 16:16 ` Junio C Hamano
2022-03-30 22:07 ` Jonathan Tan
2022-03-30 22:12 ` Josh Steadmon
2022-03-30 22:46 ` Calvin Wan
2022-03-29 20:35 ` [PATCH v3 0/3] " Junio C Hamano
2022-03-29 22:40 ` Calvin Wan
2022-03-31 1:50 ` Junio C Hamano
2022-05-02 17:08 ` [PATCH v4 0/8] cat-file: add --batch-command remote-object-info command Calvin Wan
2022-05-02 17:08 ` [PATCH v4 1/8] fetch-pack: refactor packet writing Calvin Wan
2022-05-02 17:08 ` [PATCH v4 2/8] fetch-pack: move fetch default settings Calvin Wan
2022-05-02 22:58 ` Junio C Hamano
2022-05-03 23:06 ` Jonathan Tan
2022-05-05 18:13 ` Calvin Wan
2022-05-02 17:08 ` [PATCH v4 3/8] object-store: add function to free object_info contents Calvin Wan
2022-05-02 23:23 ` Junio C Hamano
2022-05-04 19:09 ` Junio C Hamano
2022-05-05 0:15 ` Junio C Hamano
2022-05-05 16:47 ` Calvin Wan
2022-05-05 17:01 ` Junio C Hamano
2022-05-02 17:09 ` [PATCH v4 4/8] object-info: send attribute packet regardless of object ids Calvin Wan
2022-05-03 0:05 ` Junio C Hamano
2022-05-03 19:21 ` Calvin Wan
2022-05-03 23:11 ` Jonathan Tan
2022-05-02 17:09 ` [PATCH v4 5/8] transport: add client side capability to request object-info Calvin Wan
2022-05-03 0:54 ` Junio C Hamano
2022-05-03 18:58 ` Calvin Wan
2022-05-04 15:42 ` Junio C Hamano
2022-05-03 23:15 ` Jonathan Tan
2022-05-04 15:50 ` Junio C Hamano
2022-05-02 17:09 ` [PATCH v4 6/8] transport: add object-info fallback to fetch Calvin Wan
2022-05-03 23:27 ` Jonathan Tan
2022-05-02 17:09 ` [PATCH v4 7/8] cat-file: move parse_cmd and DEFAULT_FORMAT up Calvin Wan
2022-05-02 17:09 ` [PATCH v4 8/8] cat-file: add --batch-command remote-object-info command Calvin Wan
2022-05-04 21:27 ` Jonathan Tan
2022-05-05 18:13 ` Calvin Wan
2022-05-05 18:44 ` Junio C Hamano
2022-05-05 19:09 ` Junio C Hamano
2022-05-05 19:19 ` Calvin Wan
2022-07-31 15:24 ` ZheNing Hu
2022-08-08 17:43 ` Calvin Wan
2022-07-28 23:02 ` [PATCH v5 0/6] " Calvin Wan
2022-07-28 23:56 ` Junio C Hamano
2022-07-29 0:02 ` Junio C Hamano
2022-07-31 8:41 ` Phillip Wood
2022-08-04 22:57 ` Calvin Wan
2022-09-30 23:23 ` Junio C Hamano
2022-07-28 23:02 ` [PATCH v5 1/6] fetch-pack: refactor packet writing Calvin Wan
2022-07-28 23:02 ` [PATCH v5 2/6] fetch-pack: move fetch initialization Calvin Wan
2022-07-28 23:02 ` [PATCH v5 3/6] protocol-caps: initialization bug fix Calvin Wan
2022-07-29 17:51 ` Junio C Hamano
2022-07-28 23:02 ` [PATCH v5 4/6] serve: advertise object-info feature Calvin Wan
2022-07-29 17:57 ` Junio C Hamano
2022-08-01 18:28 ` Calvin Wan
2022-08-01 18:44 ` Ævar Arnfjörð Bjarmason
2022-08-01 18:47 ` Junio C Hamano
2022-08-01 18:58 ` Calvin Wan
2022-07-28 23:02 ` [PATCH v5 5/6] transport: add client support for object-info Calvin Wan
2022-07-29 18:06 ` Junio C Hamano
2022-08-04 20:28 ` Calvin Wan
2022-08-01 13:30 ` Phillip Wood [this message]
2022-08-04 22:20 ` Calvin Wan
2022-08-08 10:07 ` Phillip Wood
2022-08-01 14:26 ` Phillip Wood
2022-08-08 9:16 ` Phillip Wood
2022-07-28 23:02 ` [PATCH v5 6/6] cat-file: add remote-object-info to batch-command Calvin Wan
2022-07-29 6:25 ` Ævar Arnfjörð Bjarmason
2022-08-07 5:50 ` ZheNing Hu
2022-08-08 18:07 ` Calvin Wan
2022-08-11 10:58 ` Ævar Arnfjörð Bjarmason
2022-07-29 18:21 ` Junio C Hamano
2022-08-08 18:37 ` Calvin Wan
2022-09-28 13:12 ` Ævar Arnfjörð Bjarmason
2022-07-31 15:02 ` [PATCH v4 0/8] cat-file: add --batch-command remote-object-info command ZheNing Hu
2022-08-08 17:32 ` Calvin Wan
2022-08-13 22:17 ` Junio C Hamano
2022-02-09 19:09 ` [PATCH] fetch —object-info-format: client option for object-info John Cai
2022-02-10 22:49 ` Calvin Wan
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=b33ec478-8858-faa7-2678-f02559426539@gmail.com \
--to=phillip.wood123@gmail.com \
--cc=calvinwan@google.com \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=johncai86@gmail.com \
--cc=jonathantanmy@google.com \
--cc=philipoakley@iee.email \
--cc=phillip.wood@dunelm.org.uk \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.