polygon
October 22, 2022, 11:41am
1
Due to an accident, I’m temporarily down to one usable arm which makes my typing rather slow. Wondering if someone has some form of speech-to-text for text entry done in NixOS. I’ve found stt
but it seems rather bare and just a component of a workable solution. I’d possibly imagine like a virtual keyboard or hotkey, then I say my stuff, and then the recognized text gets typed like a virtual keyboard. Also open for other suggestions. I can do config changes but anything that still requires a lot of experimentation is out of the picture for practicability reasons.
jtojnar
October 22, 2022, 12:18pm
2
I tried packaging three different tools but I was not impressed by either of them:
NixOS:master
← jtojnar:ibus-vosk
opened 01:10PM - 05 Aug 22 UTC
###### Description of changes
Alternative to https://github.com/NixOS/nixpkgs… /pull/185148, using vosk as ibus input method, rather than a low-level input manipulator.
Alternative to https://github.com/NixOS/nixpkgs/pull/173847, using Vosk instead of Coqui stt.
<!--
For package updates please link to a changelog or describe changes, this helps your fellow maintainers discover breaking updates.
For new packages please briefly describe the package or provide a link to its homepage.
-->
###### Things done
- Built on platform(s)
- [x] x86_64-linux
- [ ] aarch64-linux
- [ ] x86_64-darwin
- [ ] aarch64-darwin
- [ ] For non-Linux: Is `sandbox = true` set in `nix.conf`? (See [Nix manual](https://nixos.org/manual/nix/stable/command-ref/conf-file.html))
- [ ] Tested, as applicable:
- [NixOS test(s)](https://nixos.org/manual/nixos/unstable/index.html#sec-nixos-tests) (look inside [nixos/tests](https://github.com/NixOS/nixpkgs/blob/master/nixos/tests))
- and/or [package tests](https://nixos.org/manual/nixpkgs/unstable/#sec-package-tests)
- or, for functions and "core" functionality, tests in [lib/tests](https://github.com/NixOS/nixpkgs/blob/master/lib/tests) or [pkgs/test](https://github.com/NixOS/nixpkgs/blob/master/pkgs/test)
- made sure NixOS tests are [linked](https://nixos.org/manual/nixpkgs/unstable/#ssec-nixos-tests-linking) to the relevant packages
- [ ] Tested compilation of all packages that depend on this change using `nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD"`. Note: all changes have to be committed, also see [nixpkgs-review usage](https://github.com/Mic92/nixpkgs-review#usage)
- [x] Tested basic functionality of all binary files (usually in `./result/bin/`)
- [22.11 Release Notes (or backporting 22.05 Release notes)](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md#generating-2211-release-notes)
- [ ] (Package updates) Added a release notes entry if the change is major or breaking
- [ ] (Module updates) Added a release notes entry if the change is significant
- [ ] (Module addition) Added a release notes entry if adding a new NixOS module
- [ ] (Release notes changes) Ran `nixos/doc/manual/md-to-db.sh` to update generated release notes
- [x] Fits [CONTRIBUTING.md](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md).
<!--
To help with the large amounts of pull requests, we would appreciate your
reviews of other pull requests, especially simple package updates. Just leave a
comment describing what you have tested in the relevant package/service.
Reviewing helps to reduce the average time-to-merge for everyone.
Thanks a lot if you do!
List of open PRs: https://github.com/NixOS/nixpkgs/pulls
Reviewing guidelines: https://nixos.org/manual/nixpkgs/unstable/#chap-reviewing-contributions
-->
NixOS:master
← jtojnar:ibus-deepspeech
opened 08:13AM - 21 May 22 UTC
###### Description of changes
Accidentally found a [Hackaday article](https:/… /hackaday.com/2018/01/17/speech-recognition-for-linux-gets-a-little-closer/) mentioning ibus-deepspeech in the bookmarks of my RSS feed reader so decided to try it out. Updated stt while at it.
I managed to make the [example from gst-deepspeech](https://github.com/Elleo/gst-deepspeech/pull/20#issuecomment-1133557850) work with upstream audio samples but not with my microphone – perhaps. The ibus engine unfortunately does not seem to do anything for me either.
**Edit:** It must be some issue with the gstreamer pipeline since when I record my voice and play it, it works :
```ShellSession
$ nix-shell -I nixpkgs=$HOME/Projects/nixpkgs -p gst_all_1.gstreamer -p gst-deepspeech --run 'gst-launch-1.0 --messages filesrc location=$PWD/fox.flac ! decodebin ! audioconvert ! audiorate ! audioresample ! deepspeech silence-length=20 ! fakesink | grep -E deepspeech.+text'
TensorFlow: v2.8.0-8-g06c8fea58fd
Coqui STT: v1.3.0-0-g148fa743
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
Got message #169 from element "deepspeech0" (element): deepspeech, timestamp=(guint64)18446744073709551615, stream-time=(guint64)18446744073709551615, running-time=(guint64)18446744073709551615, intermediate=(boolean)false, text=(string)"to\ be\ bound\ to\ jumped\ over\ the\ lazy\ dog";
```
Whereas the [pipeline from the ibus engine](https://github.com/Elleo/ibus-deepspeech/blob/3ea7c537e60e3ff66c566355c2b365febc91a91a/engine/engine.py#L45) is way too off:
```ShellSession
$ nix-shell -I nixpkgs=$HOME/Projects/nixpkgs -p gst_all_1.gstreamer -p gst-deepspeech --run 'gst-launch-1.0 --messages pulsesrc ! audioconvert ! audiorate ! audioresample ! deepspeech silence-length=20 ! fakesink | grep -E deepspeech.+text'
TensorFlow: v2.8.0-8-g06c8fea58fd
Coqui STT: v1.3.0-0-g148fa743
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
Got message #62 from element "deepspeech0" (element): deepspeech, timestamp=(guint64)18446744073709551615, stream-time=(guint64)18446744073709551615, running-time=(guint64)18446744073709551615, intermediate=(boolean)true, text=(string)"for\ the\ ";
Got message #63 from element "deepspeech0" (element): deepspeech, timestamp=(guint64)18446744073709551615, stream-time=(guint64)18446744073709551615, running-time=(guint64)18446744073709551615, intermediate=(boolean)true, text=(string)"for\ the\ wise";
Got message #64 from element "deepspeech0" (element): deepspeech, timestamp=(guint64)18446744073709551615, stream-time=(guint64)18446744073709551615, running-time=(guint64)18446744073709551615, intermediate=(boolean)true, text=(string)"for\ the\ wise";
Got message #65 from element "deepspeech0" (element): deepspeech, timestamp=(guint64)18446744073709551615, stream-time=(guint64)18446744073709551615, running-time=(guint64)18446744073709551615, intermediate=(boolean)true, text=(string)"for\ the\ wise";
…
```
<!--
For package updates please link to a changelog or describe changes, this helps your fellow maintainers discover breaking updates.
For new packages please briefly describe the package or provide a link to its homepage.
-->
###### Things done
- Built on platform(s)
- [x] x86_64-linux
- [ ] aarch64-linux
- [ ] x86_64-darwin
- [ ] aarch64-darwin
- [ ] For non-Linux: Is `sandbox = true` set in `nix.conf`? (See [Nix manual](https://nixos.org/manual/nix/stable/command-ref/conf-file.html))
- [ ] Tested, as applicable:
- [NixOS test(s)](https://nixos.org/manual/nixos/unstable/index.html#sec-nixos-tests) (look inside [nixos/tests](https://github.com/NixOS/nixpkgs/blob/master/nixos/tests))
- and/or [package tests](https://nixos.org/manual/nixpkgs/unstable/#sec-package-tests)
- or, for functions and "core" functionality, tests in [lib/tests](https://github.com/NixOS/nixpkgs/blob/master/lib/tests) or [pkgs/test](https://github.com/NixOS/nixpkgs/blob/master/pkgs/test)
- made sure NixOS tests are [linked](https://nixos.org/manual/nixpkgs/unstable/#ssec-nixos-tests-linking) to the relevant packages
- [ ] Tested compilation of all packages that depend on this change using `nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD"`. Note: all changes have to be committed, also see [nixpkgs-review usage](https://github.com/Mic92/nixpkgs-review#usage)
- [ ] Tested basic functionality of all binary files (usually in `./result/bin/`)
- [22.05 Release Notes (or backporting 21.11 Release notes)](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md#generating-2205-release-notes)
- [ ] (Package updates) Added a release notes entry if the change is major or breaking
- [ ] (Module updates) Added a release notes entry if the change is significant
- [ ] (Module addition) Added a release notes entry if adding a new NixOS module
- [ ] (Release notes changes) Ran `nixos/doc/manual/md-to-db.sh` to update generated release notes
- [x] Fits [CONTRIBUTING.md](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md).
NixOS:master
← jtojnar:nerd-dictation
opened 06:36PM - 04 Aug 22 UTC
###### Description of changes
Fixes: https://github.com/NixOS/nixpkgs/issues/… 176121
cc @guy-who-googles
<!--
For package updates please link to a changelog or describe changes, this helps your fellow maintainers discover breaking updates.
For new packages please briefly describe the package or provide a link to its homepage.
-->
###### Things done
- Built on platform(s)
- [x] x86_64-linux
- [ ] aarch64-linux
- [ ] x86_64-darwin
- [ ] aarch64-darwin
- [ ] For non-Linux: Is `sandbox = true` set in `nix.conf`? (See [Nix manual](https://nixos.org/manual/nix/stable/command-ref/conf-file.html))
- [ ] Tested, as applicable:
- [NixOS test(s)](https://nixos.org/manual/nixos/unstable/index.html#sec-nixos-tests) (look inside [nixos/tests](https://github.com/NixOS/nixpkgs/blob/master/nixos/tests))
- and/or [package tests](https://nixos.org/manual/nixpkgs/unstable/#sec-package-tests)
- or, for functions and "core" functionality, tests in [lib/tests](https://github.com/NixOS/nixpkgs/blob/master/lib/tests) or [pkgs/test](https://github.com/NixOS/nixpkgs/blob/master/pkgs/test)
- made sure NixOS tests are [linked](https://nixos.org/manual/nixpkgs/unstable/#ssec-nixos-tests-linking) to the relevant packages
- [ ] Tested compilation of all packages that depend on this change using `nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD"`. Note: all changes have to be committed, also see [nixpkgs-review usage](https://github.com/Mic92/nixpkgs-review#usage)
- [ ] Tested basic functionality of all binary files (usually in `./result/bin/`)
- [22.11 Release Notes (or backporting 22.05 Release notes)](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md#generating-2211-release-notes)
- [ ] (Package updates) Added a release notes entry if the change is major or breaking
- [ ] (Module updates) Added a release notes entry if the change is significant
- [ ] (Module addition) Added a release notes entry if adding a new NixOS module
- [ ] (Release notes changes) Ran `nixos/doc/manual/md-to-db.sh` to update generated release notes
- [x] Fits [CONTRIBUTING.md](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md).
<!--
To help with the large amounts of pull requests, we would appreciate your
reviews of other pull requests, especially simple package updates. Just leave a
comment describing what you have tested in the relevant package/service.
Reviewing helps to reduce the average time-to-merge for everyone.
Thanks a lot if you do!
List of open PRs: https://github.com/NixOS/nixpkgs/pulls
Reviewing guidelines: https://nixos.org/manual/nixpkgs/unstable/#chap-reviewing-contributions
-->
2 Likes
NobbZ
October 22, 2022, 12:24pm
3
In general there is talon
, which I managed to package in an earlier version, though not the current one. Though the old version isn’t available anymore.
It’s much more than just TTS, but a full command engine and can therefore be used for programming as well.
If someone manages to package it stably, I’d like to hear more about it!
1 Like
polygon
October 22, 2022, 12:37pm
4
I found Talon as well. Managed to get a current version to run using steam-run. Unfortunately, there’s still a lot of wrong detections and sometimes trying to correct using my voice makes everything worse. However using my remaining hand to control cursor and special keys is quite a good combination and also quite fast. I will play around with this some more and see if I am actually faster than typing one handed. Will definitely try to create a Nix package for this once I’m back to two typing hands.
1 Like
noblman
January 29, 2023, 10:18pm
5
What is the status of this. I would be interested in using nerd-dictation, but haven’t got the first clue how to get it installed.
jtojnar
January 29, 2023, 10:33pm
6
Someone needs to adopt the package and create a PR against Nixpkgs, addressing the review comments.