Rather than modify the database schema for a core table that could break inventory completely, I would recommend preforming some cleanup on your database. Running out of the int space would require having more than 2 billion inventoried files, which is an egregious number that we haven't even seen environments with over 50,000 devices worth of inventory reach.
This doc has a script that preforms such maintenance and, among other things, removes unneeded file entries from that table (i.e. ones not tied to anything and not used for any tracking).
Changing the FileInfoInstance_Idn to a bigint can cause issues with the inventory service, the console, SLM, EPS, and various other parts of the product that may be reading that value in as a signed integer in the code itself, and will throw an unhandled exception when that causes an overflow there as well. And since that table is essentially the central repository for all file information collected, there are many parts of the product that use it. These modules are designed with the constraints of the DB in mind so that value checking doesn't need to be done on every single thing we touch.