Does this project use CRLF or LF as line endings?

[Rakete]

I am confused about the line endings. I’ve been running into problems where when I try to merge upstream changes locally, they show up as “entire file has changed”, which is problematic since I can’t tell what actually changed.

So I’ve looked at a diff now, and it seems that is because all of the line endings changed. I assume that must be because git or vscode default to LF now and will replace line endings so that you end up with LF in the remote. And I just use the default config so I think my line endings must have been replaced to LF, but then that means this project would be using CRLF as line endings, is that correct?

[Rakete]

I actually suggest switching to LF, but if you want to keep using CRLF, then I would add a .gitattributes file that enforces that:

*       text=auto eol=crlf

I have been trying to fix the issue locally and its surprisingly difficult, which is why I would suggest just using LF and be done with it. But that .gitattributes file seems to fix the problem for me.

Though I am still not entirely sure if the project really does use CRLF or its somehow some configuration issue on my end. The automatic line ending replacement by git does not make it easier since I am never 100% sure if I am looking at the actual repo state or the git-replaced-the-line-endings-for-me state.

EDIT: I think the .gitattributes doesn’t actually help, this only makes it so files that are checked out have the line endings set to CRLF. So people using Linux will probably hate it.

My problem is that I have a fork that I push to, then git will by default replace CRLF → LF, and then when I merge changes into my fork I get a huge change that for the entire file because of that. I now manually replaced all files I have locally that had LF as line endings with CRLF, and then pushed those to my fork. Hopefully that resolve the issue for me.

I still think you should switch to LF, most people will just expect that and it matches the default behaviour of git. If you don’t do that I think you have to tell people to use git config core.autocrlf false when working on the source.

[Thomas Larsson]

Since I’m only using Windows these days, I have not given a thought about line endings. I don’t see any problems with using LF consistently,

I added a .gitattributes files at the repo root, with the content

* text=auto
* text eol=lf

but that doesn’t seem to change existing files. I cloned the repo to a new place but the file endings don’t seem to have changed; some files use LF and some CRLF. What am I missing here?

[Thomas Larsson]

The .gitattributes file does something, because I got an error when trying to commit a file with CRLF endings. I manually changed the endings of the python files in the main directory, but have not changed file in subdirectories yet.

[Rakete]

Using * text=auto should make sure that when people clone, they get the default behaviour for text files [1], which is it should checkout with whatever line ending they need on their operating system. This should help when people use git without setting autocrlf[2] to true, and it seems the default is indeed false. It also should produce a warning that CRLF will get converted to LF, which I think you’re already seen. I thought this only applies on checkout, but seems this is wrong and it also makes sure that when you push files the line endings are replaced with LF [3], 4]

The other line should not be needed, but will make it so when you checkout you get LF line endings. Right now I am not sure I would add that actually since the * text=auto line should work fine if I understand the docs correctly. I actually had a whole section here about setting autocrlf to true, but it seems with * text=autothat should not be needed (you can still check if its set to false, that would explain why the files seem to be CRLF in the repo, might be easier to just set it to true globally if you don’t want to deal with line endings nonsense, that will always replace the line endings, but only for you, not other people).

You may try to change the line endings in the entire project once, and then commit that as one big change so that you’re sure the remote has LF endings. With the .gitattributes present in the repo (or autocrlf set to true), you can just renormalize all line endings with: git add --renormalize .[5], that should replace all remaining CRLF line endings with LF, and stage all files that changed. You can then just check with git status what would change, and if nothing changes then I think you already have LF line endings in the remote and don’t need to do anything.

But a disclaimer: I find these line endings shenanigans with git very confusing. It tries to help but I find it makes it difficult to tell what exactly is going on. So I never really know. Maybe the problem is on my side actually.

[1] https://docs.github.com/en/get-started/getting-started-with-git/configuring-git-to-handle-line-endings

[2] https://stackoverflow.com/questions/4181870/git-on-windows-what-do-the-crlf-settings-mean

[3] https://www.git-scm.com/docs/gitattributes

[4] https://stackoverflow.com/questions/30669292/at-what-point-will-gitattribute-eol-settings-be-run

[5] https://stackoverflow.com/questions/7156694/git-how-to-renormalize-line-endings-in-all-files-in-all-revisions

[Rakete]

Yeah, I just merged again and I think you definitely need to remove the * text eol=lf line, that is just forcing all files to be treated as text with LF line endings. When I try to run git add --renormalize . it replaces line endings in the *.png files too for example.

EDIT: One trick to check line endings is to use gnu find + file + grep to check. All of which should be installed when you have the git command line installed, I think I renamed find.exe (in c:/Program Files/Git/usr/bin/) to gfind.exe to make it work in all windows shells. Otherwise you can use it from the git command line only I think.

→ whereis grep 
grep: /usr/bin/grep.exe /c/Program Files/Git/usr/bin/grep.exe /usr/share/man/man1/grep.1.gz /usr/share/info/grep.info.gz
→ whereis gfind
gfind: /c/Program Files/Git/usr/bin/gfind.exe
→ whereis file 
file: /usr/bin/file.exe /usr/share/file /c/Program Files/Git/usr/bin/file.exe /usr/share/man/man1/file.1.gz /usr/share/man/mann/file.n.gz
→ gfind . -iname "\*.py" -not -type d -exec file "{}" ";" | grep "with LF"

when you call file on a path it will output the file type, and its line ending type:

→ file node.py
node.py: Python script, ASCII text executable, with CRLF line terminators

This is CRLF, which is correct, since I am on windows and removed the * text eol=lf line.

So I think you should remove it too, and then you should be good.

You already did a commit which replaced all the line endings in the entire project, so from now the git default behaviour should make it so everyone gets the line ending that fit their operating system (if you remove the * text eol=lf line), and everything that is pushed to the repo will have LF line endings.

[Thomas Larsson]

If I understand you right, changing .gitattributes to

* text=auto eol=lf

should do the trick. Done now.

[Rakete]

Just * text=auto should be enough, this will ensure files in the repo are LF, while making it so on Windows people get CRLF on checkout, and on Linux people get LF on checkout.

With * text=auto eol=lf the text=auto still behaves the same, forcing LF in the repo, but it also forces line endings to be LF on checkout, so both Windows and Linux get LF with that.

Apparently there was also a bug around this so it wouldn’t work as expected.

https://stackoverflow.com/questions/29435156/what-will-text-auto-eol-lf-in-gitattributes-do

https://stackoverflow.com/questions/70633469/what-is-the-difference-between-text-auto-and-text-auto-eol-lf

[Thomas Larsson]

OK, I changed .gitattributes again.