Will not open file (Charset Exception when reading file)

[CodeSJS]

JabRef version

Latest development branch build (please note build date below)

Operating system

Windows

Details on version and operating system

jabref 5.6 or 5.7 on WIndows 10

Checked with the latest development build

• I made a backup of my libraries before testing the latest development version.
• I have tested the latest development version and the problem persists

Steps to reproduce the behaviour

I moved a bib file to a new location and suddenly the file will not load. I will attach the log file - it's some java error that I cannot decipher. I opened the bib file in emacs, and had emacs check the syntax, and it passed the test. I beautified the file in emacs and jabref still will not load the file.

If I remove two-thirds of the file, jabref will load the file. I thought there might be a malformed entry. By trial and error, I determined the exact stopping point. However, when I remove that entry, jabref still fails.

The problem arose with jabref 5.6. I installed jabref 5.7 and the same problem arose.

jabref opens another file without a problem.

Appendix

Paste an excerpt of your log file here

Error opening file 'C:\Users\singer.2\OneDrive - The Ohio State University\Documents\Projects\TauFibrils\Notes\Bibliography\PeptideFibrils1.bib'
java.nio.charset.MalformedInputException: Input length = 1
	at java.base/java.nio.charset.CoderResult.throwException(Unknown Source)
	at java.base/sun.nio.cs.StreamDecoder.implRead(Unknown Source)
	at java.base/sun.nio.cs.StreamDecoder.read(Unknown Source)
	at java.base/java.io.InputStreamReader.read(Unknown Source)
	at java.base/java.io.BufferedReader.fill(Unknown Source)
	at java.base/java.io.BufferedReader.read(Unknown Source)
	at java.base/java.io.FilterReader.read(Unknown Source)
	at java.base/java.io.PushbackReader.read(Unknown Source)
	at org.jabref@5.7.86/org.jabref.logic.importer.fileformat.BibtexParser.read(Unknown Source)
	at org.jabref@5.7.86/org.jabref.logic.importer.fileformat.BibtexParser.skipWhitespace(Unknown Source)
	at org.jabref@5.7.86/org.jabref.logic.importer.fileformat.BibtexParser.parseFileContent(Unknown Source)
	at org.jabref@5.7.86/org.jabref.logic.importer.fileformat.BibtexParser.parse(Unknown Source)
	at org.jabref@5.7.86/org.jabref.logic.importer.fileformat.BibtexImporter.importDatabase(Unknown Source)
	at org.jabref@5.7.86/org.jabref.logic.importer.fileformat.BibtexImporter.importDatabase(Unknown Source)
	at org.jabref@5.7.86/org.jabref.logic.importer.OpenDatabase.loadDatabase(Unknown Source)
	at org.jabref@5.7.86/org.jabref.cli.ArgumentProcessor.importAndOpenFiles(Unknown Source)
	at org.jabref@5.7.86/org.jabref.cli.ArgumentProcessor.processArguments(Unknown Source)
	at org.jabref@5.7.86/org.jabref.cli.ArgumentProcessor.<init>(Unknown Source)
	at org.jabref@5.7.86/org.jabref.gui.JabRefMain.start(Unknown Source)
	at org.jabref.merged.module@5.7.86/com.sun.javafx.application.LauncherImpl.lambda$launchApplication1$9(Unknown Source)
	at org.jabref.merged.module@5.7.86/com.sun.javafx.application.PlatformImpl.lambda$runAndWait$12(Unknown Source)
	at org.jabref.merged.module@5.7.86/com.sun.javafx.application.PlatformImpl.lambda$runLater$10(Unknown Source)
	at java.base/java.security.AccessController.doPrivileged(Unknown Source)
	at org.jabref.merged.module@5.7.86/com.sun.javafx.application.PlatformImpl.lambda$runLater$11(Unknown Source)
	at org.jabref.merged.module@5.7.86/com.sun.glass.ui.InvokeLaterDispatcher$Future.run(Unknown Source)
	at org.jabref.merged.module@5.7.86/com.sun.glass.ui.win.WinApplication._runLoop(Native Method)
	at org.jabref.merged.module@5.7.86/com.sun.glass.ui.win.WinApplication.lambda$runLoop$3(Unknown Source)
	at java.base/java.lang.Thread.run(Unknown Source)

[Siedlerchr]

Hi, thanks for the report. This seems to be some problem with determining the right charset for reading. Can you check which charset it shows in emacs? JabRef always tries to guess the charset and ideally uses UTF8
Can you send us the file so we can use it for debugging purposes? (You can send it to web@jabref.org, if you do not want to upload it:

[CodeSJS]

That was it! When I saved as utf-8 from emacs, jabref was able to open. 1. I have no idea why a file that jabref was previously happy to open suddenly acquired this problem. 2. My bad. I always thought that saving as “undecided-unix” from emacs gave simply ascii files. Evidently that does not do the trick. Thank you, Sherwin From: Christoph ***@***.***> Sent: Monday, May 30, 2022 3:21 PM To: JabRef/jabref ***@***.***> Cc: Singer, Sherwin ***@***.***>; Author ***@***.***> Subject: Re: [JabRef/jabref] Will not open file (Issue #8870) Hi, thanks for the report. This seems to be some problem with determining the right charset for reading. Can you check which charset it shows in emacs? JabRef always tries to guess the charset and ideally uses UTF8 Can you send us the file Hi, thanks for the report. This seems to be some problem with determining the right charset for reading. Can you check which charset it shows in emacs? JabRef always tries to guess the charset and ideally uses UTF8 Can you send us the file so we can use it for debugging purposes? (You can send it to ***@***.******@***.***>, if you do not want to upload it: — Reply to this email directly, view it on GitHub<https://urldefense.com/v3/__https:/github.com/JabRef/jabref/issues/8870*issuecomment-1141428789__;Iw!!KGKeukY!z65f-vdKsP-xG_wCLVruN_3dBG6_Op3_lw_CBDY28c3OjU0EFbS6Ys_qX4yib1JVScyeMiijxASZINlTwZ-WJcJUteKro32nS6_t$>, or unsubscribe<https://urldefense.com/v3/__https:/github.com/notifications/unsubscribe-auth/AJII6NBXIDR472RQVRU2MXLVMUIJTANCNFSM5XLL5PBA__;!!KGKeukY!z65f-vdKsP-xG_wCLVruN_3dBG6_Op3_lw_CBDY28c3OjU0EFbS6Ys_qX4yib1JVScyeMiijxASZINlTwZ-WJcJUteKro2FW5ctt$>. You are receiving this because you authored the thread.Message ID: ***@***.******@***.***>>

[ThiloteE]

This problem goes both ways: Emacs could have changed their "undecided unix" export.
What happens if you import or open the non-working file with older versions of JabRef?

[ThiloteE]

More info needed. Since you used One-drive, a file transfer could have destroyed some parts of your file; Old HDDs can wear out and lead to bits and bytes flipping; etc. There are a multitude of reasons why files can stop working.

• Could you please provide the faulty .bib file?
• What emacs version are you working with?

I installed Emacs 26.3 to debug this and was not able to change the encoding from UTF-8 (default) to something else. How did you do it? I tried the multi language settings... but it seems I have to enter some commands. Also, could not find the "undecided-unix" option.

From JabRef side: I remember the UTF-8 comment was removed in one of the recent versions. Might have been 5.5 or 5.6.

[CodeSJS]

I have attached the file that 5.3 will open, but 5.6 and 5.7 will not open. I’m using emacs 28.1, but the following saving procedure has not changed. Click on Options ***@***.*** Multilingual Environment ***@***.*** Set Coding System ***@***.*** For Saving This Buffer This will put you into a 1-line window at the bottom of the frame. You can type in “undecided-unix” or “utf-8” If you tab while you are typing, it will auto-complete as much as possible. There are many variations of each system. For example, after entering utf-8 keep tabbing and it will show many options like utf-8, utf-8-auto,… I just used utf-8. After that you must save the file, either from the File menu or else by typing “control-x control-s” After I saved the troublesome bib file as utf-8 in this manner, jabref 5.6 and 5.7 could open the file. I have no idea where I picked up the “undecided-unix” option, but it has been my go-to for converting Microsoft files with a control character at the end of the line to (what I thought was) a standard unix ascii file. Thanks again for your help, Sherwin From: ThiloteE ***@***.***> Sent: Tuesday, May 31, 2022 6:24 PM To: JabRef/jabref ***@***.***> Cc: Singer, Sherwin ***@***.***>; Author ***@***.***> Subject: Re: [JabRef/jabref] Will not open file (Charset Exception when reading file) (Issue #8870) More info needed. Since you used One-drive, a file transfer could have destroyed some parts of your file; Old HDDs can wear out and lead to bits and bytes flipping; etc. There are a multitude of reasons why files can stop working. Could you More info needed. Since you used One-drive, a file transfer could have destroyed some parts of your file; Old HDDs can wear out and lead to bits and bytes flipping; etc. There are a multitude of reasons why files can stop working. * Could you please provide the faulty .bib file? * What emacs version are you working with? I installed Emacs 26.3 to debug this and was not able to change the encoding from UTF-8 (default) to something else. How did you do it? I tried the multi language settings... but it seems I have to enter some commands. Also, could not find the "undecided-unix" option. — Reply to this email directly, view it on GitHub<https://urldefense.com/v3/__https:/github.com/JabRef/jabref/issues/8870*issuecomment-1142694596__;Iw!!KGKeukY!1iKBqVBGqaMD85diiNR53fQt0hQS6PUmbapxUJRVcPOtjSCT_6X8QT4UnFBc93suopG_1nK4aeB2Cwsxtb3ixhSm9DhrADkilEcy$>, or unsubscribe<https://urldefense.com/v3/__https:/github.com/notifications/unsubscribe-auth/AJII6NAL5FOUM7AVATZNVATVM2GO7ANCNFSM5XLL5PBA__;!!KGKeukY!1iKBqVBGqaMD85diiNR53fQt0hQS6PUmbapxUJRVcPOtjSCT_6X8QT4UnFBc93suopG_1nK4aeB2Cwsxtb3ixhSm9DhrABaTSn8m$>. You are receiving this because you authored the thread.Message ID: ***@***.******@***.***>>

[ThiloteE]

Thank you that helped a lot.

Two things:

• I was not able to reproduce. Doing as you suggested for a small library of mine and saving in undecided-unix with Emacs 26.3 still allowed me to open the file with JabRef 5.7 development version. So I suspect it really is something specific in your library.

• You tried sending the library-file via e-mail, but GitHub does not support receiving files apparently. I think you will have to login into GitHub and upload the file directly. If it is sensitive information you also can send it to web@jabref.org

[CodeSJS]

I will have to email file to web@jabref.org. github will not accept a bib file.

[CodeSJS]

I will make the subject line attn ThiloteE

[Siedlerchr]

You can simply add .txt or something similar, e.g. .bib.txt will be accepted by github

[ThiloteE]

So I was able to reproduce. Could not open your file.

At which line do you think is the error?

Btw.:

The coding systems unix, dos, and mac are aliases for undecided-unix, undecided-dos, and undecided-mac, respectively. These coding systems specify only the end-of-line conversion, and leave the character code conversion to be deduced from the text itself.

Source: https://www.gnu.org/software/emacs/manual/html_node/emacs/Coding-Systems.html

Btw. my text editor (xed 3.2.2) also was not able to detect the encoding of the file.