video: fbdev: i740fb: Check the argument of i740_calc_vclk()

Vulnerability Description

In the Linux kernel, the following vulnerability has been resolved: video: fbdev: i740fb: Check the argument of i740_calc_vclk() Since the user can control the arguments of the ioctl() from the user space, under special arguments that may result in a divide-by-zero bug. If the user provides an improper 'pixclock' value that makes the argumet of i740_calc_vclk() less than 'I740_RFREQ_FIX', it will cause a divide-by-zero bug in: drivers/video/fbdev/i740fb.c:353 p_best = min(15, ilog2(I740_MAX_VCO_FREQ / (freq / I740_RFREQ_FIX))); The following log can reveal it: divide error: 0000 [#1] PREEMPT SMP KASAN PTI RIP: 0010:i740_calc_vclk drivers/video/fbdev/i740fb.c:353 [inline] RIP: 0010:i740fb_decode_var drivers/video/fbdev/i740fb.c:646 [inline] RIP: 0010:i740fb_set_par+0x163f/0x3b70 drivers/video/fbdev/i740fb.c:742 Call Trace: fb_set_var+0x604/0xeb0 drivers/video/fbdev/core/fbmem.c:1034 do_fb_ioctl+0x234/0x670 drivers/video/fbdev/core/fbmem.c:1110 fb_ioctl+0xdd/0x130 drivers/video/fbdev/core/fbmem.c:1189 Fix this by checking the argument of i740_calc_vclk() first.

References

• https://git.kernel.org/stable/c/59cefb583c984c0da8cf21a4c57d26d5a20dff5c
• https://git.kernel.org/stable/c/656689cb03ada4650016c153346939a1c334b1ae
• https://git.kernel.org/stable/c/d2d375eb68b4b8de6ea7460483a26fa9de56b443
• https://git.kernel.org/stable/c/2b7f559152a33c55f51b569b22efbe5e24886798
• https://git.kernel.org/stable/c/4b20c61365140d432dee7da7aa294215e7b900d9
• https://git.kernel.org/stable/c/e740e787f06671455b59d1e498c9945f7b4e7b3b
• https://git.kernel.org/stable/c/f350812e2d15278f1d867eeb997407782234fb3c
• https://git.kernel.org/stable/c/40bf722f8064f50200b8c4f8946cd625b441dda9