BigFix Troubleshoot - BFIVR - IVR Data Not Found

Running into IVR Data Not Found Error?

Here is what has caused this for you, and after that, the fix. Luckily, if you have the right credentials to do all of this, it is about a 30 min fix. This is for integrations on Tenable.io, Tenable.sc and Qualys.

My theory is that this is caused by running an Insights ETL on older versions of Insights within WebUI and then ultimately trying to plug in vulnerability scan data into the older insights database making it to where the data gets placed in outdated areas within the DB. Ultimately, WebUI gets hung up on this and then it'll no longer be able to find the IVR data for it to be represented in WebUI.

The fix... Stop WebUI service, drop the Insights database out of SQL, run two lines of commands against the BFEnterprise database to clear up Insights entries, do some clean up (if not rip and replace on the IVR service itself which may be a good idea if your IVR service is out of date) on the IVR directories, setup Insights again, do an ETL with Insights and wait for that to complete, re-initialize schemas for IVR and then run the IVR service again. I know that was a lot, I have a step by step below.

One of the best things to do before setting up insights in general is to make sure you've processed all the application updates within WebUI which can be run from the top right corner of the WebUI > gear icon > Application Updates and make sure they have all been processed.

Here is what it’ll look like if you have updates to process:

Here are the more detailed steps:

• Before beginning, I would highly recommend doing a backup of your databases/database servers where both the Insights database resides and where the BFEnterprise database resides be that traditional SQL backup or a snapshot.
• Stop WebUI Service on the server hosting WebUI.

• Open SSMS, log in with an account with the rights to drop the database (with stopping all existing connections) and DELETE the Insights Database. Yes, if you have historical data in here, you'll be starting over (sorry).

• That close existing connections option is at the bottom of the screen.
• Using the same or a new local session of SSMS (depending on where all the databases are), connect to the SQL server that is hosting the BFEnterprise database and run these queries to get coordinated and to ultimately remove two entries from the DB. Use the commands below by right-clicking on the BFEnterprise database > New Query.

select * from dbo.webui_data

delete from webui_data where App = 'Insights'

delete from webui_data where App = 'Ivr'        

• Should look like this:

• Once you it looking like the above screenshot, Click the "Execute" to the right of where it says the database name.

• Once you’ve seen two rows affected then you should be good to proceed
• From there startup the WebUI service:

• Wait for it to get up and running and then configure Insights again using the same gear icon in the top right corner in the WebUI > Insights.
• You’ll be prompted as if it was a brand-new installation at this point

• Create the new insights database indicating where you want the database to reside and then providing the credentials to create that database. The database can be created/maintained using either SQL login or AD Auth. There will be an alias box there too but that is simply a label, call that whatever you’d like. I normally recommend calling it Insights Database.
• Here is what the next step will look like, adding a datasource for Insights:

• Add the datasource for insights to gather content from will be the next step. This will be your BFEnterprise database. You’ll be providing information for what SQL server that database lives on, the credential rights to READ from it. This must be a SQL auth account. Again there will be an alias box there as well which again is just a label, I commonly guide people to call this one BigFix Root Server or something to that tune.
• Once you get that coordinated and it successfully connects it’ll give you a list of sites from BigFix that you want to pull in. I’d recommend bringing in only sites that have patch content/CVE data within it to minimize the ETL process to only what is both necessary and what will be able to be correlated within the insights for vulnerability process. Here is what that looks like:

• Once that data is all added, you can trigger the ETL process and wait for that to complete. You’ll click on the date under the “next data sync” column and then you’ll be able to kick off the ETL manually.

• The prior screen will load up giving a status. Wait for that status to go from “Running” to “Complete”.
• Once the ETL is finished you’ll have just a few last things to do. If you didn’t uninstall/reinstall BigFix IVR service then these are the steps you need. If you’re setting up IVR fresh use this article: https://help.hcltechsw.com/bigfix/11.0/integrations/Ecosystem/Install_Config/c_deploy_IVR_2.html.
• Go to the system hosting the IVR service, stop the service and then open an elevated command prompt to the directory holding the IVR service itself and call up this command BFIVR.exe --InitializeSchemas.

• This’ll prompt you for the same SA level credentials you used to setup the Insights database and place the appropriate schemas back within the database for the service to populate vulnerability data to. Once you provide those credentials you should be overall ready to go. I do recommend running BFIVR.exe --ValidateConfiguration as a good check to make sure that it thinks all the items are configured correctly and we’re ready to go.

• Next you’ll want to go into that same directory you ran the command prompt executions from and clear out some files within that directory; anything that starts with the phrasing “RecordCache”.

• Once all of that is cleared up, go to services and start up that BFIVR service once again and wait for the logs to populate under the logs directory within the BFIVR installation directory. You’ll see 3-4 files there depending on the integration you’re working with but the DataFlow log directories are the most important. Wait for those to say that the “Dataflow Execution Completed in (timer value)” and once you see those you’re ready to move on.
• Once the prior has completed, jump back to WebUI > gear in the top right corner > Insights (likely provide credentials again just to validate you’re allowed back in there) > go to the tab that says “IVR Access”. From there, in the bottom left corner you should see the option to toggle on the IVR data and an auto-configure button immediately to the right of the word “Grant”

• Once the auto-configure comes back with a green check that happens once you click it; I’ve learned to wait about 5 min for the data to populate at a minimum and then go see if you have the data you’d expect under Apps > IVR from the top left corner of WebUI.