2021-09-23 16:34:59 +08:00
|
|
|
Name: sentencepiece
|
2023-05-12 15:44:26 +08:00
|
|
|
Version: 0.1.98
|
|
|
|
|
Release: 1
|
2021-09-23 16:34:59 +08:00
|
|
|
Summary: An unsupervised text tokenizer and detokenizer
|
|
|
|
|
License: Apache-2.0
|
|
|
|
|
URL: https://github.com/google/sentencepiece
|
|
|
|
|
Source0: https://github.com/google/sentencepiece/archive/v%{version}.tar.gz
|
2021-09-29 15:10:12 +08:00
|
|
|
BuildRequires: gcc-c++ gcc autoconf pkgconfig protobuf-compiler protobuf
|
2021-09-23 16:34:59 +08:00
|
|
|
BuildRequires: cmake >= 3.14.0
|
2021-09-29 15:10:12 +08:00
|
|
|
Requires: protobuf protobuf-compiler
|
2021-09-23 16:34:59 +08:00
|
|
|
|
|
|
|
|
%description
|
|
|
|
|
SentencePiece is an unsupervised text tokenizer and detokenizer mainly for Neural Network-based text generation
|
|
|
|
|
systems where the vocabulary size is predetermined prior to the neural model training.
|
|
|
|
|
|
|
|
|
|
%global debug_package %{nil}
|
|
|
|
|
|
|
|
|
|
%prep
|
|
|
|
|
%autosetup -p1 -n %{name}-%{version}
|
|
|
|
|
|
|
|
|
|
%build
|
|
|
|
|
mkdir -p cmake/build
|
|
|
|
|
cd cmake/build
|
|
|
|
|
cmake ../../ -DCMAKE_CXX_FLAGS="-D_FORTIFY_SOURCE=2 -O2 -Wno-unused-result -Wno-sign-compare"\
|
|
|
|
|
-DCMAKE_C_FLAGS="-D_FORTIFY_SOURCE=2"\
|
|
|
|
|
-DCMAKE_BUILD_TYPE=Release\
|
|
|
|
|
-DSPM_USE_BUILTIN_PROTOBUF=ON\
|
|
|
|
|
-DSPM_ENABLE_SHARED=OFF\
|
2021-09-29 15:10:12 +08:00
|
|
|
-DSPM_BUILD_TEST=ON\
|
2021-09-23 16:34:59 +08:00
|
|
|
-DCMAKE_INSTALL_LIBDIR=%{buildroot}%{_libdir}\
|
|
|
|
|
-DCMAKE_INSTALL_BINDIR=%{buildroot}%{_bindir}\
|
|
|
|
|
-DCMAKE_INSTALL_INCDIR=%{buildroot}%{_includedir}
|
|
|
|
|
|
|
|
|
|
make -j24 V=1
|
2021-09-29 15:10:12 +08:00
|
|
|
make CTEST_OUTPUT_ON_FAILURE=1 test
|
2021-09-23 16:34:59 +08:00
|
|
|
cd ../../
|
|
|
|
|
|
|
|
|
|
%install
|
|
|
|
|
cd cmake/build
|
|
|
|
|
make install
|
2023-05-12 15:44:26 +08:00
|
|
|
sed -i'' -e "s,%{buildroot},," %{buildroot}%{_libdir}/pkgconfig/sentencepiece.pc
|
|
|
|
|
sed -i'' -e "s,${prefix}/lib,%{_libdir}," %{buildroot}%{_libdir}/pkgconfig/sentencepiece.pc
|
2021-09-23 16:34:59 +08:00
|
|
|
|
|
|
|
|
%files
|
|
|
|
|
%defattr(-,root,root)
|
|
|
|
|
%{_bindir}/spm_*
|
|
|
|
|
%{_libdir}/*.a
|
|
|
|
|
%{_libdir}/pkgconfig/*
|
2021-11-27 19:30:16 +08:00
|
|
|
%{_includedir}/sentencepiece*.h
|
2021-09-23 16:34:59 +08:00
|
|
|
|
|
|
|
|
%changelog
|
2023-05-12 15:44:26 +08:00
|
|
|
* Fri May 12 2023 liuyongqi <liuyongqi5@huawei.com> - 0.1.98-1
|
|
|
|
|
- Sentencepiece version updata: Upgraded from 0.1.92 to 0.1.98
|
|
|
|
|
|
2021-11-27 19:30:16 +08:00
|
|
|
* Fri Nov 27 2021 xiefangqi <xiefangqi2@huawei.com> - 0.1.92.6
|
|
|
|
|
- Fix split_digits support to SentencepieceTrainer spec parser
|
|
|
|
|
- Add sentencepiece.pc install
|
|
|
|
|
- Add spm_train --help option
|
|
|
|
|
- Fix FTBFS problem on armel/mips/powerpc/m68k/sh4
|
|
|
|
|
- Fix endian problem on android plarform
|
|
|
|
|
- Fix pb protobuf header file can't find problem
|
|
|
|
|
- Restore the sentence boundary
|
2023-05-12 15:44:26 +08:00
|
|
|
|
2021-11-16 19:50:09 +08:00
|
|
|
* Tue Nov 16 2021 xiefangqi <xiefangqi2@huawei.com> - 0.1.92.5
|
|
|
|
|
- add README.md/README.en.md
|
2023-05-12 15:44:26 +08:00
|
|
|
|
2021-11-02 15:50:59 +08:00
|
|
|
* Tue Nov 2 2021 xiefangqi <xiefangqi2@huawei.com> - 0.1.92-4
|
|
|
|
|
- fix of an unattainable condition
|
2023-05-12 15:44:26 +08:00
|
|
|
|
2021-11-02 11:38:21 +08:00
|
|
|
* Tue Nov 2 2021 xiefangqi <xiefangqi2@huawei.com> - 0.1.92-3
|
|
|
|
|
- Prevent Zero Width Joiner replaced with whitespace
|
2023-05-12 15:44:26 +08:00
|
|
|
|
2021-09-29 15:10:12 +08:00
|
|
|
* Wed Sep 29 2021 xiefangqi <xiefangqi2@huawei.com> - 0.1.92-2
|
|
|
|
|
- add test cases
|
2023-05-12 15:44:26 +08:00
|
|
|
|
2021-09-23 16:34:59 +08:00
|
|
|
* Wed Sep 23 2021 xiefangqi <xiefangqi2@huawei.com> - 0.1.92-1
|
|
|
|
|
- package init
|
