Volume files updated outside a container sometimes lose hardlink inside. #9524

madeline-holland · 2022-06-02T02:42:38Z

Description

When using git checkout -- <file> on a file passed to a container, the file on the inside of the container loses its hardlink, breaking things unexpectedly.

This is most prominent when working with a configuration that is passed in, like Drupal's composer.json which may need to be refreshed from a repository.
Interestingly, in the original repository I was testing this in moving the file does not make it lose its link when moved back, while with my test script (below) it does.
Running git checkout -- <file> from a shell outside the container inside a folder that is passed through into the container does not produce this behavior.
A workaround to fix the link is to restart the container with docker compose restart or similar.

Steps to reproduce the issue:

git clone this repository: https://github.com/madeline-holland/docker-bug-test
Run test.sh

Without using my test repository:

Create a docker-compose.yml with a file mounted in the container as a volume.
Run the container.
Modify the file outside with git checkout -- <file> or (somethimes) moving the file to a different folder and back.
Observe that the file inside loses its hardlink ls -alh column 2 = 0 rather than 1.

Describe the results you received:
When updating a file outside a container which is passed through with git or sometimes moving it, the linked version inside loses its hardlink.

Describe the results you expected:
Upon updating a file outside the container, the linked version inside would likewise be updated, like when editing it with nano or another editor.

Additional information you deem important (e.g. issue happens only occasionally):
This seems similar to this issue and this one too.

Output of docker compose version:

Docker Compose version v2.5.0

Output of docker info:

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  app: Docker App (Docker Inc., v0.9.1-beta3)
  buildx: Docker Buildx (Docker Inc., v0.8.2-docker)
  compose: Docker Compose (Docker Inc., v2.5.0)
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc., 0.6.0)
  scan: Docker Scan (Docker Inc., v0.17.0)

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 3
 Server Version: 20.10.16
 Storage Driver: btrfs
  Build Version: Btrfs v5.16.2
  Library Version: 102
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 212e8b6fa2f44b9c21b2798135fc6fb7c53efc16
 runc version: v1.1.1-0-g52de29d
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
  cgroupns
 Kernel Version: 5.17.9-300.fc36.x86_64
 Operating System: Fedora Linux 36 (Workstation Edition)
 OSType: linux
 Architecture: x86_64
 CPUs: 16
 Total Memory: 14.54GiB
 Name: <redacted>
 ID: <redacted>
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Additional environment details:
See this repository for more details.

My main question is this:

It seems that this is caused by Docker not knowing the file has changed and automatically re-linking it, is there a way to manually recreate the hardlink? Running docker compose restart can take quite a while when there's multiple images, so a less intensive solution would be quite helpful.

The text was updated successfully, but these errors were encountered:

maxcleme · 2022-06-03T06:55:27Z

@madeline-holland

Thanks for reporting this.

I've tried to replicate the issue on my end with these using latest compose (2.6.0) :

compose.yaml

services:
  a:
    build: .
    volumes:
      - ./foo:/app/foo

Dockerfile

FROM alpine:latest
WORKDIR /app
COPY entrypoint.sh .
CMD source entrypoint.sh

entrypoint.sh

#!/bin/sh

while true;
do
  ls -alh /app/foo;
  sleep 1;
done;

I then tried to move the file back and forth while the container was running :

test-compose-9524-a-1  | -rw-r--r--    1 root     root          12 Jun  3 06:44 /app/foo
test-compose-9524-a-1  | -rw-r--r--    1 root     root          12 Jun  3 06:44 /app/foo
test-compose-9524-a-1  | -rw-r--r--    1 root     root          12 Jun  3 06:44 /app/foo
test-compose-9524-a-1  | -rw-r--r--    1 root     root          12 Jun  3 06:44 /app/foo
test-compose-9524-a-1  | -rw-r--r--    1 root     root          12 Jun  3 06:44 /app/foo
test-compose-9524-a-1  | -rw-r--r--    1 root     root          12 Jun  3 06:44 /app/foo
test-compose-9524-a-1  | ls: /app/foo: No such file or directory
test-compose-9524-a-1  | ls: /app/foo: No such file or directory
test-compose-9524-a-1  | ls: /app/foo: No such file or directory
test-compose-9524-a-1  | ls: /app/foo: No such file or directory
test-compose-9524-a-1  | ls: /app/foo: No such file or directory
test-compose-9524-a-1  | ls: /app/foo: No such file or directory
test-compose-9524-a-1  | -rw-r--r--    1 root     root          12 Jun  3 06:44 /app/foo
test-compose-9524-a-1  | -rw-r--r--    1 root     root          12 Jun  3 06:44 /app/foo
test-compose-9524-a-1  | -rw-r--r--    1 root     root          12 Jun  3 06:44 /app/foo
test-compose-9524-a-1  | -rw-r--r--    1 root     root          12 Jun  3 06:44 /app/foo

Using git checkout -- foo has the same output, am I missing something ?

madeline-holland · 2022-06-03T18:07:02Z

Hello,

On my end, using your setup, I ended up with the following:

docker-test-a-1  | -rw-rw-r--    1 1000     1000           7 Jun  3 17:53 /app/foo
docker-test-a-1  | -rw-rw-r--    1 1000     1000           7 Jun  3 17:53 /app/foo
docker-test-a-1  | -rw-rw-r--    0 1000     1000           7 Jun  3 17:53 /app/foo
docker-test-a-1  | -rw-rw-r--    0 1000     1000           7 Jun  3 17:53 /app/foo

Moving the file manually out of the folder and back in (with mv or copy-paste in Nautilus) keeps the link (top two lines).

I then ran the following commands:

git init
git commit -m "x"
echo "Hi" >> foo
git checkout -- foo

Which then broke the link (second to third line).

Running git checkout without a change in the file doesn't modify the file, and doesn't break the link, but when there is a change it does reset the file and break the link.

Thank you for your help with this!

ndeloof · 2022-12-13T13:39:22Z

Please double check this issue doesn't also apply when you run a plain docker run container with a comparable setup. In such circumstances, issue is not related to compose, and should be transferred to github.com/docker/docker

stale · 2023-06-18T05:22:48Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

stale bot added the stale label Jun 18, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Volume files updated outside a container sometimes lose hardlink inside. #9524

Volume files updated outside a container sometimes lose hardlink inside. #9524

madeline-holland commented Jun 2, 2022

maxcleme commented Jun 3, 2022

madeline-holland commented Jun 3, 2022

ndeloof commented Dec 13, 2022

stale bot commented Jun 18, 2023

Volume files updated outside a container sometimes lose hardlink inside. #9524

Volume files updated outside a container sometimes lose hardlink inside. #9524

Comments

madeline-holland commented Jun 2, 2022

maxcleme commented Jun 3, 2022

compose.yaml

Dockerfile

entrypoint.sh

madeline-holland commented Jun 3, 2022

ndeloof commented Dec 13, 2022

stale bot commented Jun 18, 2023