Microphone, Camera, Sound

I have now spent a lot of time investigating the issue with iFrames and the problem with …

-allowing microphone (mic-activation)
-allowing sound playback (sound-activation)
-allowing camera (camera-activation)

…inside the HTML-Component.

Since there are no options given inside the VELO-APIs for that…
…only → allow-fullscreen is given…


…so it is not really possible to access microphone or activate sound and camerea inside an iFrame.

At this point the question → WHY?

Reading post about that mentioned issue → i recognize that there are not many answers.

Wix do not want to provide such possibility?

On other hand, there would be still the possibility to use the CUSTOM-ELEMENT, but seems also not to be very promissing, since it would also have to request permission from inside of the element and the allawens would have to be setted from outside of the CUSTOM-ELEMENT itself as i understood.

Another possibility? —> NPM ? …waht else…?

There are workarounds, which are not really good, or even do not really work.

Someone has some infos about that ?

Edit: Ok, it seems that the main-Wix-iFrame has already set all allowens to tru (almost all) right from the beginning (on load) …
2024-01-11 14_49_47-TEST _ WixWorld

How i could get now my own iFrame, which is sitting inside Wix’s own iFrame?
Or another question: How could i provide allowense of mic to the sub-iFrame, once i have found it?

Other ideas or approaches?

Hey Dima,

Each of these should be accessed directly through the DOM, which can be done through custom elements and some deeper coding.

Here’s an of how to setup the code for speech recognition on your website, through the native DOM API for the microphone. https://youtu.be/utNVp2MDVoo

First of all, thank you Chris for your very informative response. You are one of those who can help me with my specific questions.

Ok, i checked your version with the LixVoice, it’s more promissing then this one (which is by the way absolutely NOT USERFRIENDLY)…

But if i am honest, i have already generated something what has been working, until i changed something, what changed everything (and the bad thing is, i do not know anymore what i have changeg, so i can’t use the HTML-Component anymore at moment) to do the same thing, like shown in your EXAMPLE, but using some custom generated speech2text environment (html-component) and → elevenlabs.

So in detail, my setup is:

HTML-Component with some custom code inside, which listen to my mic.
It was working of somekind like…

(sorry for the bad voice-equalizer on screen → was just a test) :joy:

Now you can see the spoken text in realtime inside of my HTML-Component.
Somehow i was able to activate that function and i got my speech to text inside the html in realtime, that means you were able to see each character like typed by a machine, but in real controlled by speech.

So, till here everything was working → WAS :woozy_face: :face_with_diagonal_mouth: :face_with_diagonal_mouth:

So i know, that it is possible to access through HTML-Component.
I just did not realize what i have done or what i have changed, so it was working.

So my questions at this point, regarding my setup…
a) Was the HTML-Component designed to allow microphone, camera, vr, speaker?
Because such thinngs are not included inside of the HTML-Component-API.
b) What would change, if i would use a CUSTOM-ELEMENT instead? Would it be easier for me to get those functions?
c) I like to use the HTML-Comp. because i am very familiar with this element right now, but i would switch over to CUSTOM-ELEMENT if this element provides more functionalty with less restrictions.

I also have seen some workarounds on how to give my own CUSTOM-ELEMENT the needed → allowens <— (inside of the origin-page-code) which did not really work well in most of cases.

The whole concept is connected to the following post…

d) How much → realtime ← is your version ?
e) what exactly would be possible if i would switch over to use your proposed version with lix-voice?
f) Would i be able for example just open a dropdown by my voice on my screen (this is what i am capable to do with my version)
g) Would i be able to create a new element on my screen, again just by my voice, i don’t think so. Yes, of course everything is running inside of my HTML-Element i know, but this is not a big problem, because at the end for real results i still can postMessage to get what i want on my Wix website.
h) And one more question about your provided version. Which languages does it support? (multi-lingual, like mine) ?

So, hopefully you recognize that i would have boundaries, if i would go your way.
It gives me some possibilities, but do not offer me the full package i need.

So for me it is important to be able to get that working on an HTML-Comp or custom element.

A third option maybe would be → NPM-package, but here again → NPM would directly run on my wix-page → again i wouldn’t have all the mentioned possibilities to get creative, since for example you CAN NOT create a new element by code, like .appendChild() → you know what i mean.

You also would not be able to reposition an element by speech on your site (maybe only if you would implement wix-animations somehow) .

And surely i could mention mcuh more situations, where your presented solution (good one), would not work for me (not flexible enough).

Or do you have other thoughts? If so → please let me know.

Best regards Dima

1 Like

Ok, got my old version to work back again!
Thanks anyway Chris! D a m n, that took me a long time. :rofl:

Additionaly:
And i mentioned elevenLabs, which i choose for generating → speech2text (so the backward of text2speech).

Here one of videos → https://www.youtube.com/watch?v=t52Bi-ZUZjA which prooves the quality and possibilities of ElevenLabs - text2speech - capabilities.

So puting all together again, you will understand my soon ready JARVIS :rofl: (once it is ready :face_with_raised_eyebrow:)

Ok, wanted to come back one more time to this topic.

Since anybody gave me a real answer to all my questions, i will answer all of them now by my own.

It took me nor for about 3-days to investigate and discover the HTML-Component.
Sice i like this HTML-Component and like already mentioned before → i am now very familiar ← with it, i use it in most cases, as soon as it comes to using HTML-Code, CSS-Code, or generating owm CUSTOM-STUFF on my website.

But the HTML-Component, do not provide all possibilities and is someking of limited.
Some facts about the HTML-Component.

ADVANTAGES:

  • easy implementation of HTML-Code
  • easy implementation of CSS-Code
  • gives you the possibility to create your own stuff (elements), like buttons, menus, dropdowns, carousels, slideshows and all other known and even unknowns elements.
  • gives you the ability to communicate between your wix-page and the HTML-Component.
  • gives you the ability to show in FULLSCREEN
  • gives you the ability to include websites
  • and surely some more…

DISATVANTAGES:

  • a lot of restrictions, like …
    a) microphone --------> not allowed
    b) speaker --------------> not allowed
    c) camera ---------------> not allowed
    d) vr (virtual-reality) → not allowed
    e) and yes even fetch and http-functions, did not really work (but not sure, maybe also my own fault and i did some wrong, but did not work for me)
  • and almost forgot to mention, that the HTML-Component has even more restrictions, when you don’T have a premium wix-website.

Unfortunately, this resctrictions reduces attractiveness of the HTML-Component and because all this disatvantages (mentioned here) are also not mentioned inside of the VELO-API → that means Wix do not provide (or at least do not want to privide) such options, regarding the HTML-Component.

Well, it’s really a pitty!

But as we know every system has it’s weakness (wix is not an exception), once you found it → you start to like the HTML-Component even more. :rofl: (jail & jail-break-princip)

@ Chris-Derell: (thanks one more time at this piont, i appreciate always your thoughts!)

During my investigations of the HTML-Component, i had more then twice the thoughts to start to use your provided version, but at the end, i was able to make it work the way i wanted.

And i also made more thoughts about the following points…
g) Would i be able to create a new element on my screen, again just by my voice, i don’t think so. Yes, of course everything is running inside of my HTML-Element i know, but this is not a big problem, because at the end for real results i still can postMessage to get what i want on my Wix website.
h) And one more question about your provided version. Which languages does it support? (multi-lingual, like mine) ?

…and you know what, i think it would be even possible to do so, even with your provided version (not directly, but it would still work indirectly).

Conclusion, i would be able to generate my autonomous JARVIS-PROJECT, in both of cases → (TEXT-2-SPEECH & SPEECH-2-TEXT / REQUEST-REPLY-PRINCIP).