I understand the problem. The arrow is large relative to the size of the text.
The only solution is to increase the height and width of the input.
As a designer, I would recommend you do so. This is a better user experience.
It is difficult for a user to press such a small button on a phone.
I would increase the button height to at least 48px and no more than 2 buttons in the same row.
This will leave enough space for the text and the user will be able to clearly see the content of the input and press the button. https://material.io/components/text-fields#specs