this fixes a regression in a previous patch, this time
also taking account for the different cache locations.
all of get.sh needs to be purged, and re-written clean.
it looks clean. but it's years of hacks.
Signed-off-by: Leah Rowe <leah@libreboot.org>
Don't hardcode the cache directory, and don't store
remotes anymore. This change retains compatibility
in practice, with the older directory location, because
it's extremely unlikely that newly generated locations
would conflict with old ones.
With this new change, non-mirror git clone caches are
now done twice; one directory per remote, rather than
one directory with two remotes.
This is just inherently much more reliable.
Signed-off-by: Leah Rowe <leah@libreboot.org>
Some repositories might use CR-LF line endings. This option
keeps Git from mangling patches when merging.
Repositories that don't do this, such as ALL repositories
currently used by xbmk, will be unaffected by this change.
This is being done in preparation for importing MrChromebox
edk2, as Intel's own edk2 repository on GitHub uses these.
Signed-off-by: Leah Rowe <leah@libreboot.org>
Don't do one repository for all remotes. Do one *clone* per
remote.
This also means that users no longer download information twice,
in practice, because the backup repository will only be downloaded
if the main one didn't work.
Theoretically, this change is makes the process less efficient, but
in practise it's more reliable now.
We do now use --mirror on the git clone command for caches, but we
already did git pull --all before.
This just ensures that we absolutely have all local code.
NOTE:
The new code isn't used by default. To use it, you must do:
export XBMK_CACHE_MIRROR="y"
Otherwise, the old behaviour will continue to be used. This is
because the new code, while correct, puts more strain on upstream
servers (more code being downloaded), and can result in higher amounts
of disk space being used. The old behaviour wasn't broken, so we'll
also support that method.
TODO: perhaps also have a check in place to re-use both caches,
where available, regardless of XBMK_CACHE_MIRROR?
Signed-off-by: Leah Rowe <leah@libreboot.org>
whatchanged is deprecated, and results in an error
on modern git versions, prompting you to include
the --i-still-use-this argument
what absolute, utter fucking arrogance. i use the
whatchanged feature every fucking day.
i will be complaining to git-scm.com about this.
but that's what we do in libreboot. we adapt.
Signed-off-by: Leah Rowe <leah@libreboot.org>
xbmkdir checks if a directory exists, before running
mkdir, and then still uses -p
i was testing xbmk on arch linux today, and noticed
that it errored out when a directory already exists.
i'm mitigating against buggy or differently behaving
mkdir implementations this way, by wrapping around
it.
Signed-off-by: Leah Rowe <leah@libreboot.org>
the --status flag seems to be a GNUism
as stated in the previous commit, i import sbase
suckless now, so as to have a consistent implementation
of sha512sum.
this ensures that its output is reliable, when i'm using
the output of this command within backticks.
Signed-off-by: Leah Rowe <leah@libreboot.org>
this allows me to remove several eval calls, and the
errors relating to configs can now show exactly which
function they occured in, allowing for easier debugging.
once again, eval should be used sparingly if at all.
Signed-off-by: Leah Rowe <leah@libreboot.org>
i will eventually find a way to remove them all,
while still leaving the code completely clean.
in practise, i never use the contents of a file
for eval and the inputs are carefully checked.
however, over-use of eval is always a bad idea
in shell scripting.
Signed-off-by: Leah Rowe <leah@libreboot.org>
use of ./mk -F behaves the same as -f before the
previous commit.
this can be useful, during development when we want
to update revisions.
Signed-off-by: Leah Rowe <leah@libreboot.org>
we pull from upstream in cached git repos, before performing
an operation, and we run from the cache, but we do this every
time, even if a local revision exists, defeating the purpose
of the caching; on unreliable/intermittent internet connections,
this can cause a problem.
this also causes us problems with gnulib.git and grub.cfg, which
for *some reason* are really slow, even when doing a pull.
this change improves the efficiency of the build system, during
release builds, on a development repository where we already
have lots of caches.
Signed-off-by: Leah Rowe <leah@libreboot.org>
the check for whether a file is present is unnecessary,
because the following cp command would also print the
file name if it doesn't exist, and exit with the same
non-zero status.
let cp do the work.
Signed-off-by: Leah Rowe <leah@libreboot.org>
the previous function name was misleading, because
this tries multiple methods including git and curl.
therefore, this was renamed to match what it dose.
Signed-off-by: Leah Rowe <leah@libreboot.org>
use the new functionality in err(), whereby a given
function name and arguments can be provided, for
debugging purposes.
something similar was already done in a few places,
and replaced with this unified functionality.
this patch will make xbmk much easier to debug, under
fault conditions.
Signed-off-by: Leah Rowe <leah@libreboot.org>
the newer way handles escaped characters better, and it
can be nested more easily. it's also more readable.
personally, i prefer the old way, because it's more
minimalist, but it occurs to me that a lot of people
nowadays don't know about backticks, but they do know
of the modern way.
to make the code more readable, i have modernised it.
Signed-off-by: Leah Rowe <leah@libreboot.org>
similar to the previous failed patch, which tried to
also generate it each time, but that led to issues.
this version of the same change merely maintains the
current hardcoding logic, while putting it in xbtmp.
that way, it's more robustly cleared upon exit from
the parent instance of xbmk.
this also reduces the chance of race conditions,
since it's in a unique place each time, rather than
going in XBMK_CACHE.
Signed-off-by: Leah Rowe <leah@libreboot.org>
and generate them, don't hardcode them - this reduces
the chance of race conditions, which we have seen in
the past and which current execution flow in xbmk even
mitigates in a few places, by doing things in a certain
order.
this change makes the code more robust and easier to
maintain.
Signed-off-by: Leah Rowe <leah@libreboot.org>
we no longer separate them. xbloc was the on-disk
tmp directory, whereas xbtmp used to be in /tmp
which we assumed to be tmpfs (it may not be, but
often is on many workstation setups - and our
documentation recommended doing this).
as mentioned in the previous commit, benchmarking
shows little speed difference using tmpfs /tmp
versus on-disk /tmp, for our purposes at least.
therefore, the handling of tmp files is being
greatly simplified.
Signed-off-by: Leah Rowe <leah@libreboot.org>
the actual code works fine, but it's quite hacky.
there are times when use of eval is acceptable; this
is not one of those times, but i'd used it in this
instance when i was being a bit crazy about code size
reductions during my audits.
Signed-off-by: Leah Rowe <leah@libreboot.org>
stick it in git_prep, which both single- and multi-tree
projects will use, when downloading git repositories.
Signed-off-by: Leah Rowe <leah@libreboot.org>
The following execution will result in another printf
that says exactly what is being downloaded.
There is no need to inform the user twice about
what is being downloaded.
Signed-off-by: Leah Rowe <leah@libreboot.org>
A git-pull is performed immediately after git-fetch.
Git-pull already performs git-fetch as a prerequisite.
Signed-off-by: Leah Rowe <leah@libreboot.org>
We rely on a non-zero exit on other try_ commands, which
works fine there because we then check the file afterward
and error out accordingly.
For git repositories, we assume that both mirrors are
identical and therefore once we get to the first clone
attempt, we assume that it must succeed.
Therefore, if it does not succeed, we must fail. This fixes
a regression I found in testing, where sometimes a failed
patching attempt would not result in an error exit, and
would therefore result in broken sources being present.
In practise, I always very closely watch the terminal when
testing xbmk, especially when updating project patches, so
we probably didn't introduce any broken sources in practice.
Signed-off-by: Leah Rowe <leah@libreboot.org>
Right now, if cache/clone/PROJECT/ already exists,
the logic for pulling new changes doesn't execute,
and neither does the logic for updating remotes.
This is bad when updating revisions, because then
manual updating is required, defeating the purpose
of xbmk's own automation in this regard.
Fix it by only checking the cached download on files,
not Git repositories; the try_git function itself will
already perform this check, before updating remotes
and pulling in new commits from upstream.
The updating only happens when a given target directory
doesn't exist, e.g. src/flashprog/ or src/grub/default/,
so this won't slow down release builds for example.
Signed-off-by: Leah Rowe <leah@libreboot.org>
Leah Rowe