Weird failure when building on Raspbian / Debian 10.11

[sigkill]

Getting this error upon attempting to run:

`go get github.com/golang/snappy

github.com/golang/snappy

asm: 00001 (/root/go/src/github.com/golang/snappy/encode_arm64.s:30) TEXT "".emitLiteral(SB), NOSPLIT, $32-56: unaligned frame size 32 - must be 8 mod 16 (or 0)
asm: 00119 (/root/go/src/github.com/golang/snappy/encode_arm64.s:264) TEXT "".encodeBlock(SB), $32896-56: unaligned frame size 32896 - must be 8 mod 16 (or 0)
asm: assembly failed`

[nigeltao]

That is weird. What does go env say for you?

CC'ing @AWSjswinney and @cuonglm for arm64 assembly things.

[cuonglm]

@sigkill what's go version do you use?

[AWSjswinney]

I tested using the debian:buster container image which is using go 1.11.6. I can reproduce the problem there. It looks like that version of go requires the frame size to be 8 mod 16 to accommodate a link register. I did a little digging through the commit history and it looks like that requirement changed starting in go 1.12.

The frame sizes are currently 0 mod 16 but we can add 8 bytes to each function to make room for the link register and make this code backward compatible with 1.11 (and earlier?).

diff --git a/encode_arm64.s b/encode_arm64.s
index f8d54ad..f0c876a 100644
--- a/encode_arm64.s
+++ b/encode_arm64.s
@@ -27,7 +27,7 @@
 // The unusual register allocation of local variables, such as R10 for the
 // source pointer, matches the allocation used at the call site in encodeBlock,
 // which makes it easier to manually inline this function.
-TEXT ·emitLiteral(SB), NOSPLIT, $32-56
+TEXT ·emitLiteral(SB), NOSPLIT, $40-56
        MOVD dst_base+0(FP), R8
        MOVD lit_base+24(FP), R10
        MOVD lit_len+32(FP), R3
@@ -261,7 +261,7 @@ extendMatchEnd:
 // "var table [maxTableSize]uint16" takes up 32768 bytes of stack space. An
 // extra 64 bytes, to call other functions, and an extra 64 bytes, to spill
 // local variables (registers) during calls gives 32768 + 64 + 64 = 32896.
-TEXT ·encodeBlock(SB), 0, $32896-56
+TEXT ·encodeBlock(SB), 0, $32904-56
        MOVD dst_base+0(FP), R8
        MOVD src_base+24(FP), R7
        MOVD src_len+32(FP), R14

I did just a basic go test on this, and it works on 1.11 and 1.17, but I haven't done any further testing. @nigeltao would you like to include a change like this for backward compatibility?

[sigkill]

Thank you for checking, I had been using what was supplied with the Ubuntu, Debian and Raspbian distros, but I removed and installed from the golang website instead.

The versions were:

From Debian, seems okay:

cat /etc/debian_version
11.0
go version
go version go1.15.9 linux/amd64

From Ubuntu:

lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 20.04.3 LTS
Release:	20.04
Codename:	focal

apt info golang-go
Package: golang-go
Version: 2:1.13~1ubuntu2
Priority: optional
Section: devel
Source: golang-defaults
Origin: Ubuntu
Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>
Original-Maintainer: Go Compiler Team <team+go-compiler@tracker.debian.org>
Bugs: https://bugs.launchpad.net/ubuntu/+filebug
Installed-Size: 53.2 kB
Depends: golang-1.13-go, golang-src (>= 2:1.13~1ubuntu2)
Recommends: golang-race-detector-runtime
Suggests: git
Conflicts: gccgo-go
Breaks: gccgo-5 (<< 5.3.1-12~), gccgo-6 (<< 6-20160319-1~), golang-doc (<< 2:1.6.1+1~), golang-go-darwin-386, golang-go-darwin-amd64, golang-go-freebsd-386, golang-go-freebsd-amd64, golang-go-freebsd-arm, golang-go-linux-386, golang-go-linux-amd64, golang-go-linux-arm, golang-go-netbsd-386, golang-go-netbsd-amd64, golang-go-windows-386, golang-go-windows-amd64, golang-go.tools (<< 1:0.0~git20151026.0.0f9d71c-1~), golang-golang-x-tools (<< 1:0.0~git20151026.0.0f9d71c-1~)
Replaces: golang-doc (<< 2:1.6.1+1~), golang-go-darwin-386, golang-go-darwin-amd64, golang-go-freebsd-386, golang-go-freebsd-amd64, golang-go-freebsd-arm, golang-go-linux-386, golang-go-linux-amd64, golang-go-linux-arm, golang-go-netbsd-386, golang-go-netbsd-amd64, golang-go-windows-386, golang-go-windows-amd64, golang-go.tools (<< 1:0.0~git20151026.0.0f9d71c-1~), golang-golang-x-tools (<< 1:0.0~git20151026.0.0f9d71c-1~)
Homepage: https://golang.org
Download-Size: 22.0 kB
APT-Manual-Installed: yes
APT-Sources: http://us.archive.ubuntu.com/ubuntu focal/main amd64 Packages
Description: Go programming language compiler, linker, compiled stdlib
 The Go programming language is an open source project to make programmers more
 productive. Go is expressive, concise, clean, and efficient. Its concurrency
 mechanisms make it easy to write programs that get the most out of multicore
 and networked machines, while its novel type system enables flexible and
 modular program construction. Go compiles quickly to machine code yet has the
 convenience of garbage collection and the power of run-time reflection. It's a
 fast, statically typed, compiled language that feels like a dynamically typed,
 interpreted language.
 .
 This is a dependency package that depends on the default Go toolchain and
 compiled standard library. It is only built on architectures that have a
 native toolchain ("gc compiler"). Packages that want to build with whichever
 of gc or gccgo is available should depend on golang-any.

[sigkill]

The golang from the website appears to be okay, but the issue appears to have been with the package within one or two of the systems I have running Ubuntu, Raspbian, or Debian. The arm and amd64 versions from the golang site appear to work fine, and install without this issue.

[AWSjswinney]

That looks like an amd64 platform which wouldn't see this problem at all, since it's for arm64 only. Presumably you are using a Raspberry Pi since you're using Raspbian. Can you check there?

[sigkill]

AWSjswinney

I believe the golang-go package from Debian Buster is the same version that one of the arm based systems with the issue, prior to downloading a compiled tarball from the golang website, uninstalling the Debian package, and installing the golang website's version.

[sigkill]

Aha!

Found one of the hosts that had an issue:

Golang, installed from website, no issues:

root@raspberrypi-0001:~# which go
/usr/local/go/bin/go
root@raspberrypi-0001:~# go version
go version go1.17.5 linux/arm64

The package from Raspbian appears to be the version with the issue:

root@raspberrypi-0001:~# apt info golang-go
Package: golang-go
Version: 2:1.11~1
Priority: optional
Section: devel
Source: golang-defaults
Maintainer: Go Compiler Team <team+go-compiler@tracker.debian.org>
Installed-Size: 52.2 kB
Depends: golang-1.11-go, golang-src (>= 2:1.11~1)
Conflicts: gccgo-go
Breaks: gccgo-5 (<< 5.3.1-12~), gccgo-6 (<< 6-20160319-1~), golang-doc (<< 2:1.6.1+1~), golang-go-darwin-386, golang-go-darwin-amd64, golang-go-freebsd-386, golang-go-freebsd-amd64, golang-go-freebsd-arm, golang-go-linux-386, golang-go-linux-amd64, golang-go-linux-arm, golang-go-netbsd-386, golang-go-netbsd-amd64, golang-go-windows-386, golang-go-windows-amd64, golang-go.tools (<< 1:0.0~git20151026.0.0f9d71c-1~), golang-golang-x-tools (<< 1:0.0~git20151026.0.0f9d71c-1~)
Replaces: golang-doc (<< 2:1.6.1+1~), golang-go-darwin-386, golang-go-darwin-amd64, golang-go-freebsd-386, golang-go-freebsd-amd64, golang-go-freebsd-arm, golang-go-linux-386, golang-go-linux-amd64, golang-go-linux-arm, golang-go-netbsd-386, golang-go-netbsd-amd64, golang-go-windows-386, golang-go-windows-amd64, golang-go.tools (<< 1:0.0~git20151026.0.0f9d71c-1~), golang-golang-x-tools (<< 1:0.0~git20151026.0.0f9d71c-1~)
Homepage: https://golang.org
Tag: devel::TODO, devel::buildtools, devel::compiler, role::program
Download-Size: 23.4 kB
APT-Sources: http://deb.debian.org/debian buster/main arm64 Packages
Description: Go programming language compiler, linker, compiled stdlib
 The Go programming language is an open source project to make programmers more
 productive. Go is expressive, concise, clean, and efficient. Its concurrency
 mechanisms make it easy to write programs that get the most out of multicore
 and networked machines, while its novel type system enables flexible and
 modular program construction. Go compiles quickly to machine code yet has the
 convenience of garbage collection and the power of run-time reflection. It's a
 fast, statically typed, compiled language that feels like a dynamically typed,
 interpreted language.
 .
 This is a dependency package that depends on the default Go toolchain and
 compiled standard library. It is only built on architectures that have a
 native toolchain ("gc compiler"). Packages that want to build with whichever
 of gc or gccgo is available should depend on golang-any.

[sigkill]

As you stated, it appears to be based on buster:

root@raspberrypi-0001:~# cat /etc/debian_version
10.11
root@raspberrypi-0001:~# cat /etc/ap
apparmor.d/ apt/
root@raspberrypi-0001:~# cat /etc/apt/sources.list
deb http://deb.debian.org/debian buster main contrib non-free
deb http://deb.debian.org/debian-security/ buster/updates main contrib non-free
deb http://deb.debian.org/debian buster-updates main contrib non-free
# Uncomment deb-src lines below then 'apt-get update' to enable 'apt-get source'
#deb-src http://deb.debian.org/debian buster main contrib non-free
#deb-src http://deb.debian.org/debian-security/ buster/updates main contrib non-free
#deb-src http://deb.debian.org/debian buster-updates main contrib non-free

[sigkill]

I did get the error on non-arm based distros as well, am trying to find the system that also had the issue, but it may have been an Alpine based host.

[AWSjswinney]

The package from Raspbian appears to be the version with the issue:

Yes, that's consistent with my findings. Your options are:

1. Use a go distribution from the website. Anything 1.12 or later should be fine, but 1.16 and 1.17 both have other arm64 optimizations, so I recommend the latest version.
2. Manually patch snappy with the diff I posted above.
3. Wait for a fix to be published by Snappy (if the project decided backward compatibility is important).

[sigkill]

Thank you! I've gone with the website version for now, but very much appreciate all the effort and hard work to correct the issue and find the compatibility diff -- very cool!

[sigkill]

@sigkill what's go version do you use?

Sorry, I should have posted these details when I opened up the issue. Thank you nigeltao and cuonglm as well.

[nigeltao]

I pushed @AWSjswinney's suggestion (from a comment above). I don't have any arm64 devices handy, so if somebody could verify that it still works, I'll tag a v0.0.5.

I also assume that the $0-48 fragments below are still good (and don't need changing), even though 0 is not 8 (mod 16). Let me know if they also need updating.

$ grep ^TEXT *arm64.s
decode_arm64.s:TEXT ·decode(SB), NOSPLIT, $56-56
encode_arm64.s:TEXT ·emitLiteral(SB), NOSPLIT, $40-56
encode_arm64.s:TEXT ·emitCopy(SB), NOSPLIT, $0-48
encode_arm64.s:TEXT ·extendMatch(SB), NOSPLIT, $0-48
encode_arm64.s:TEXT ·encodeBlock(SB), 0, $32904-56