From: Simon Glass <sjg@chromium.org>
To: Yannic Moog <Y.Moog@phytec.de>
Cc: "alpernebiyasak@gmail.com" <alpernebiyasak@gmail.com>,
Benjamin Hahn <B.Hahn@phytec.de>,
"trini@konsulko.com" <trini@konsulko.com>,
"festevam@gmail.com" <festevam@gmail.com>,
Yashwanth Varakala <Y.Varakala@phytec.de>,
"upstream@lists.phytec.de" <upstream@lists.phytec.de>,
"tharvey@gateworks.com" <tharvey@gateworks.com>,
"u-boot@lists.denx.de" <u-boot@lists.denx.de>,
Teresa Remmet <T.Remmet@phytec.de>
Subject: Re: [PATCH RFC 1/6] tools: binman: ftest.py: fake ext blobs per default
Date: Fri, 14 Feb 2025 06:48:53 -0700 [thread overview]
Message-ID: <CAFLszTgdM3U3foHUKsXQY3SSs0-Gb1MpJOsW0KB+t1Wt=NUsWg@mail.gmail.com> (raw)
In-Reply-To: <9a0969442915d922caedeb9fa9dd7c56dc3fa1ed.camel@phytec.de>
Hi Yannic,
On Fri, 14 Feb 2025 at 00:18, Yannic Moog <Y.Moog@phytec.de> wrote:
>
> Hi Simon,
>
> On Thu, 2025-02-13 at 07:01 -0700, Simon Glass wrote:
> > Hi Yannic,
> >
> > On Thu, 13 Feb 2025 at 00:15, Yannic Moog <Y.Moog@phytec.de> wrote:
> > >
> > > Hi Simon,
> > >
> > > On Mon, 2025-02-10 at 06:06 -0700, Simon Glass wrote:
> > > > Hi Yannic,
> > > >
> > > > On Wed, 29 Jan 2025 at 03:30, Yannic Moog <y.moog@phytec.de> wrote:
> > > > >
> > > > > The top level Makefile calls binman with fake-ext-blobs.
> > > > > The test setup should reflect this to spot potential bugs before
> > > > > reaching users.
> > > > >
> > > > > Signed-off-by: Yannic Moog <y.moog@phytec.de>
> > > > > ---
> > > > > tools/binman/ftest.py | 2 +-
> > > > > 1 file changed, 1 insertion(+), 1 deletion(-)
> > > > >
> > > > > diff --git a/tools/binman/ftest.py b/tools/binman/ftest.py
> > > > > index a553ca9e564..1f194f9ecae 100644
> > > > > --- a/tools/binman/ftest.py
> > > > > +++ b/tools/binman/ftest.py
> > > > > @@ -372,7 +372,7 @@ class TestFunctional(unittest.TestCase):
> > > > > def _DoTestFile(self, fname, debug=False, map=False, update_dtb=False,
> > > > > entry_args=None, images=None, use_real_dtb=False,
> > > > > use_expanded=False, verbosity=None, allow_missing=False,
> > > > > - allow_fake_blobs=False, extra_indirs=None, threads=None,
> > > > > + allow_fake_blobs=True, extra_indirs=None, threads=None,
> > > > > test_section_timeout=False, update_fdt_in_elf=None,
> > > > > force_missing_bintools='', ignore_missing=False, output_dir=None):
> > > > > """Run binman with a given test file
> > > > >
> > > > > --
> > > > > 2.43.0
> > > > >
> > > >
> > > > I would like this default to stay the same (False) because that is the
> > > > normal case for Binman. We should expand the test-coverage as needed.
> > >
> > > Can you help me understand why we fake blobs in the call to binman when building U-Boot?
> > > You say that not faking is the normal case so why are we doing something abnormal in the top-
> > > level
> > > Makefile?
> >
> > I mean that not faking is the normal case for Binman's code base. Yes,
> > U-Boot requests faking.
> >
> > >
> > > I think that this inconsistency is a potential for bugs going unnoticed so whatever we decide to
> > > do
> > > I would very much like to be consistent with testing and "normal" builds.
> >
> > Binman is tested by its own tests which handle both cases, so we don't
> > have a test gap there.
>
> Sorry, I am still stumped here. I feel like my question as to why U-Boot requests faking has not
> been answered above. Can you please explain that again?
U-Boot needs to use fakes largely because in some cases the fake is
passed to mkimage, which sometimes cannot operate at all if there is a
missing or zero-sized file. So we create a fake just so that the build
will succeed (with warnings).
>
> And I am left a bit confused since I thought these RFC patches show that there indeed was a test gap
> (in a single test case testFitFirmwareLoadables only, but still) due to not faking external blobs.
> Is it not possible that there are other potential problems in regards to faking blobs that we have
> not discovered, yet?
Yes, please fix the test gap and if there are other tests we need to
add, we should add them. The goal of Binman is 100% test coverage. But
faked blobs are more of an unfortunate workaround, not the primary
goal of Binman. So I would like tests to explicitly select faked blobs
when needed, rather than having this on everywhere. It should only
affect a very small amount of tests which deal with the pain of
external blobs.
Regards,
Simon
next prev parent reply other threads:[~2025-02-14 13:49 UTC|newest]
Thread overview: 28+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-01-29 10:29 [PATCH RFC 0/6] Fix handling of optional blobs in binman Yannic Moog
2025-01-29 10:29 ` [PATCH RFC 1/6] tools: binman: ftest.py: fake ext blobs per default Yannic Moog
2025-02-10 13:06 ` Simon Glass
2025-02-13 7:15 ` Yannic Moog
2025-02-13 14:01 ` Simon Glass
2025-02-14 7:18 ` Yannic Moog
2025-02-14 13:48 ` Simon Glass [this message]
2025-02-17 7:07 ` Yannic Moog
2025-01-29 10:29 ` [PATCH RFC 2/6] tools: binman: ftest: pass allow_fake_blob to _DoReadFileDtb Yannic Moog
2025-02-10 13:09 ` Simon Glass
2025-01-29 10:29 ` [PATCH RFC 3/6] tools: binman: ftest: fix tests that require non-faked ext blobs Yannic Moog
2025-02-10 13:09 ` Simon Glass
2025-02-13 7:15 ` Yannic Moog
2025-01-29 10:29 ` [PATCH RFC 4/6] binman: fix faked optional entry handling Yannic Moog
2025-02-10 13:09 ` Simon Glass
2025-01-29 10:29 ` [PATCH RFC 5/6] binman: test: assert optional blobs don't cause non-functionality Yannic Moog
2025-02-10 13:07 ` Simon Glass
2025-01-29 10:29 ` [PATCH RFC 6/6] binman: doc: update Optional entries Yannic Moog
2025-02-10 13:08 ` Simon Glass
2025-02-13 7:21 ` Yannic Moog
2025-02-13 14:01 ` Simon Glass
2025-02-14 7:05 ` Yannic Moog
2025-02-14 13:48 ` Simon Glass
2025-02-17 7:21 ` Yannic Moog
2025-02-17 13:13 ` Simon Glass
2025-02-18 13:15 ` Yannic Moog
2025-02-19 0:01 ` Simon Glass
2025-02-10 13:08 ` [PATCH RFC 0/6] Fix handling of optional blobs in binman Simon Glass
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAFLszTgdM3U3foHUKsXQY3SSs0-Gb1MpJOsW0KB+t1Wt=NUsWg@mail.gmail.com' \
--to=sjg@chromium.org \
--cc=B.Hahn@phytec.de \
--cc=T.Remmet@phytec.de \
--cc=Y.Moog@phytec.de \
--cc=Y.Varakala@phytec.de \
--cc=alpernebiyasak@gmail.com \
--cc=festevam@gmail.com \
--cc=tharvey@gateworks.com \
--cc=trini@konsulko.com \
--cc=u-boot@lists.denx.de \
--cc=upstream@lists.phytec.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.