Update fork with latest changes from Google master #16
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
fix appveyor python 3.9 build
Resolves google#603
…ntroduction_to_readme Update README.md
Add missing #include for BYTE_ORDER
add spm proto headers to install
Prevent Zero Width Joiner replaced with whitespace
Example: sid=1313136 left=37596 right=37597 produce 18446744071878513373, which is incorrect.
Remove @pkgconfiglibs@ and @pkgconfigcflags@ which were being replaced with the empty string because these variables were never set anywhere.
When configured to use external protobuf, remove -lprotobuf-lite from pkg-config file's Libs section. The sentencepiece libraries link with libprotobuf-lite; it is not necessary for users of the sentencepiece libraries to link with protobuf unless they are using static linking so this should have been in Libs.private. Use Requires.private so that we use protobuf's pkg-config file to get the flags.
Fix overlinking with protobuf
Fix nasty bug in BPE position encoding
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.