So, I just cannot.. Quite inflexible.
I've seen both this question and documentation, but I didn't saw it say explicitly you can't do it (tho tbh I didn't read it very carefully, as I was just scanning for concrete information).
I will not "just live with .import files", because they are in wrong place.
I might just write script to symlink these images for me, so more like second option.
Copying only images that I use would also work, but it is exactly same effort as symlink-ing only images that I use, which is what I did till this time.