portable: fixes for NetBSD * Move the main incldue/got_compat.h to the top of the files they're included in. * Define SHA256Init and friends to the system equivalents.

portable: configure: split out dependencies Rather than assume all dependencies are required for all programs, split them out. This will make packaging easier, as well as splitting the code to use subprojects. Note that due to the use of config.h semantics, in most cases the got_compat.h header file is now at the top of the .c file it is included in, so that it can handle the system header inclusion properly.

remove now unused sha1 handling from inflate and deflate both were changed to use the new got_hash APIs.

add some helper functions to compute hashes This adds a set of functions to abstract over SHA1Init, SHA1Update, SHA1Final, their respective SHA256 variants and how to compare digests. Replace all the SHA1*() usage with the new APIs. It's a preparatory step for sha256 handling. ok stsp@

portable: rework SHA detection Simply the SHA detection by not predicating on libcrypto, but instead checking individual header files.

portable: remove sha1.h; found portably Remove sha1.h as this is found portably across systems.

include sha2.h too where sha1.h is included In preparation for wide sha256 support; stsp@ agrees. Change done mechanically with find . -iname \*.[cy] -exec sam {} + X ,x/<sha1\.h>/i/\n#include <sha2.h>

zero-initialize all of struct got_deflate_buf in got_deflate_init() Fixes a potential bogus free on error. got_deflate_end() relies on zb.flags to be initialized. It will try to free a bogus zb.outbuf pointer if zb.flags happens to set by uninitialized stack memory while we are calling got_deflate_end() in an error path.

introduce gotd(8), a Git repository server reachable via ssh(1) This is an initial barebones implementation which provides the absolute minimum of functionality required to serve got(1) and git(1) clients. Basic fetch/send functionality has been tested and seems to work here, but this server is not yet expected to be stable. More testing is welcome. See the man pages for setup instructions. The current design uses one reader and one writer process per repository, which will have to be extended to N readers and N writers in the future. At startup, each process will chroot(2) into its assigned repository. This works because gotd(8) can only be started as root, and will then fork+exec, chroot, and privdrop. At present the parent process runs with the following pledge(2) promises: "stdio rpath wpath cpath proc getpw sendfd recvfd fattr flock unix unveil" The parent is the only process able to modify the repository in a way that becomes visible to Git clients. The parent uses unveil(2) to restrict its view of the filesystem to /tmp and the repositories listed in the configuration file gotd.conf(5). Per-repository chroot(2) processes use "stdio rpath sendfd recvfd". The writer defers to the parent for modifying references in the repository to point at newly uploaded commits. The reader is fine without such help, because Git repositories can be read without having to create any lock-files. gotd(8) requires a dedicated user ID, which should own repositories on the filesystem, and a separate secondary group, which should not have filesystem-level repository access, and must be allowed access to the gotd(8) socket. To obtain Git repository access, users must be members of this secondary group, and must have their login shell set to gotsh(1). gotsh(1) connects to the gotd(8) socket and speaks Git-protocol towards the client on the other end of the SSH connection. gotsh(1) is not an interactive command shell. At present, authenticated clients are granted read/write access to all repositories and all references (except for the "refs/got/" and the "refs/remotes/" namespaces, which are already being protected from modification). While complicated access control mechanism are not a design goal, making it possible to safely offer anonymous Git repository access over ssh(1) is on the road map.

make got_pack_create() write to a file descriptor instead of a stdio FILE The old code required a seekable output file. This conflicts with requirements of future gotd(8), which will write pack file data to network sockets. ok op@

portable: add back sys/queue.h Now that the handling of including sys/queue.h is better, there's no need to remove those lines from the source. Copy the location of those original sys/queue.h lines from upstream at the same line number, so as to avoid any conflicts in the future.

store deltas in compressed form while packing, both in memory and cache file This reduces memory and disk space consumption during packing. with tweaks + memleak on error fix from op@ ok op@

fix infinite loop in got-index-pack for pack files >= 4GB in size Because of a missing range check our zlib wrapper would end up calling zlib over and over with zero bytes of input. Problem reported by semarie and naddy. Fixed with help from millert@. ok millert naddy

compress delta data from delta_cache directly into pack file

map raw object files into memory while packing if possible

match the unsigned char type used by the zlib interface ok stsp

portable: add FreeBSD support This adds the capability to compile got-portable on FreeBSD.

portable: initial Linux compilation This commit modifies the GoT main branch to be able to compile it under linux.

simplify deflate checksumming code; fixes errors with upcoming 'gotadmin pack' tweak + ok naddy

add checksum support to got_deflate_to_file() This will eventually be used by 'gotadmin pack'. Checksum init and finalization will need to be done by the caller since many objects will be written out in compressed form while we are computing checksums across the entire pack file. ok millert, naddy

rename got_error_prefix_errno() to got_error_from_errno()

got_error_from_errno -> got_error_prefix_errno also add got_error_prefix_errno2 and got_error_prefix_errno3 which should hopefully all be merged into a single function with variadic args (but can't alloc mem)

make got path APIs available to library consumers

flush compressed output in got_deflate_read()

add some initial code for zlib compression