Course: Microsoft Power BI Training
Topic: Understanding Power BI Data Refresh
Course Link: Microsoft Power BI Training
Demo Link: Register for Course Demo
JustAcademy Website: Visit JustAcademy
Power BI Data Refresh is the process of updating data used by a Power BI report or semantic model so that users can analyze recent information from the original data sources. Data may change frequently in Excel files, SQL databases, cloud applications, SharePoint lists, and other systems. Refresh helps Power BI incorporate those changes where the connection and storage mode require it.
For example, a retail company may receive new sales transactions every day. If its Power BI report uses imported data, the report may continue displaying previously loaded sales figures until the semantic model is refreshed successfully. Refreshing the data allows the report to reflect the newly available transactions.
Data refresh is an important part of business intelligence because outdated information can lead to incorrect conclusions, delayed decisions, and inaccurate performance measurements.
Data refresh is the process through which Power BI updates data or related content from its source or synchronizes supported resources. The exact process depends on the storage mode, connection type, source system, and refresh operation being performed.
During a typical import-mode refresh, Power BI connects to the source, retrieves the required data, applies relevant transformations, processes the semantic model, and makes the updated data available to report queries after successful completion.
Suppose a company maintains the following sales records in a database:
| Order ID | Product | Sales Amount |
|---|---|---|
| 101 | Laptop | ₹50,000 |
| 102 | Monitor | ₹15,000 |
| 103 | Keyboard | ₹2,000 |
The initial total sales amount is ₹67,000. Later, the source system receives another transaction worth ₹10,000.
| Stage | Total Sales |
|---|---|
| Before the new transaction is refreshed into an imported model | ₹67,000 |
| After a successful refresh includes the new transaction | ₹77,000 |
This example assumes that the new transaction is available in the source and is included by the model's query and refresh process.
Power BI connects to different data sources to collect and analyze information. The type of source influences authentication, connectivity, transformation, refresh configuration, and performance.
The source contains the original information, while the semantic model provides the analytical structure used by Power BI reports. A semantic model can contain tables, relationships, measures, calculated columns, and other model definitions.
In Import mode, data is copied from the source into the semantic model. In DirectQuery mode, report queries generally retrieve data from the underlying source instead of importing all the source rows into the model.
Changes in source data do not always require the same type of refresh. New records in an imported table typically require data refresh. Changes to source table structures, such as renamed or removed columns, may require a schema update and changes to the model or report.
Understanding whether a change affects the data, the model structure, or the report visuals helps determine the correct troubleshooting approach.
Power BI supports several refresh-related operations. These operations serve different purposes, and they should not be treated as interchangeable.
Data refresh updates data loaded into an imported semantic model from its underlying source. It can be initiated manually or scheduled in Power BI Service when supported.
For example, refreshing an imported sales model can load the latest transactions from a SQL database.
Schema refresh synchronizes the structure of a model with supported changes in the underlying source, such as new columns or changes to column definitions.
Schema changes must be handled carefully because renamed or removed columns can affect measures, relationships, report visuals, and transformations.
Visual refresh updates report visuals by executing the relevant queries again. The results depend on the storage mode and the freshness of the underlying data.
For DirectQuery tables, visuals query the source according to the connection and caching behavior. For imported tables, refreshing visuals alone does not retrieve new source rows that have not been loaded into the semantic model.
Scheduled refresh automatically initiates supported refresh operations at configured times in Power BI Service.
It is useful when reports need to reflect regular business updates without someone manually refreshing the semantic model each day.
Manual refresh is initiated by a user or authorized process when an update is needed immediately. It is commonly used during testing, troubleshooting, and verification.
When Power BI content is connected to supported files stored in OneDrive or SharePoint Online, Power BI can synchronize the relevant file-based content. This is different from a data refresh that queries a database and reloads imported data.
Dashboard tiles may have their own cache-refresh behavior. Updating the underlying semantic model and updating a dashboard tile are related but distinct operations. The timing depends on the storage mode and the type of refresh performed.
| Refresh Type | Purpose | Typical Use |
|---|---|---|
| Data refresh | Updates imported data | Load new sales transactions |
| Schema refresh | Synchronizes supported structural changes | Recognize new source columns |
| Visual refresh | Re-executes queries used by report visuals | Update displayed results |
| Scheduled refresh | Starts refresh operations automatically at configured times | Daily business reporting |
| Manual refresh | Starts an on-demand refresh | Testing and troubleshooting |
| OneDrive refresh | Synchronizes supported file-based content | Update content linked to cloud files |
| Dashboard tile refresh | Updates dashboard tile results or caches | Display recent dashboard information |
Storage modes determine how Power BI stores or accesses data and therefore influence refresh behavior.
In Import mode, Power BI loads data into the semantic model. Queries from reports are generally answered using the imported data rather than directly querying the original source for every interaction.
Advantages:
Limitations:
DirectQuery allows Power BI to query the underlying data source when users interact with report visuals. The source remains responsible for providing query results.
Advantages:
Limitations:
DirectQuery generally does not require a conventional imported-data refresh to load source rows, but report visuals, caches, metadata, or other model elements may still require refresh-related operations.
A live connection connects a report to an existing semantic model or supported analytical model. The report primarily uses the existing model rather than maintaining an independent imported copy of its data.
In this scenario, data freshness depends on the underlying model and source. The report may still require visual or file synchronization operations.
Direct Lake is a storage mode in Microsoft Fabric that allows supported semantic models to work with data stored in OneLake without using the same traditional import process as Import mode.
Its refresh and framing behavior differs from conventional imported models. Model framing, source changes, and the available Fabric features influence how updated data becomes visible.
A composite model combines multiple storage modes, such as Import and DirectQuery. Different tables within the same model may therefore have different data-freshness and refresh requirements.
| Storage Mode | Where Data Is Accessed | Typical Refresh Consideration |
|---|---|---|
| Import | Data is loaded into the semantic model | Refresh imported data to incorporate source changes |
| DirectQuery | Queries are sent to the source | Source queries, caching, and connectivity affect freshness |
| Live connection | Uses an existing model or analytical service | Underlying model and visual refresh behavior matter |
| Direct Lake | Works with supported data in OneLake | Framing and model-specific operations affect data visibility |
| Composite | Combines storage modes | Different tables may need different refresh approaches |
Power BI Desktop allows developers to refresh data while creating and testing reports. This is useful for checking calculations, validating new records, and confirming that transformations work as expected.
Power Query Editor is used to connect to sources and apply transformations. If you change a query, such as removing columns or changing data types, select Close & Apply to apply the changes and load the resulting data into the model where applicable.
Examples of Power Query transformations include:
| Operation | Purpose |
|---|---|
| Refresh | Updates data and, in Desktop, can synchronize the model schema as part of the refresh operation. |
| Close & Apply | Applies pending Power Query changes and loads the resulting data into the model. |
| Save | Saves the current report and model changes to the PBIX file. |
| Publish | Transfers supported content to Power BI Service. |
These operations serve different purposes. Saving a file does not by itself refresh the source data, and publishing does not automatically guarantee that a future refresh schedule has been configured.
Power BI Service provides options for managing supported semantic models after they are published. Depending on the model and user permissions, these options can include manual refresh, scheduled refresh, credentials, gateway connections, and refresh history.
The exact menu labels can vary by interface. Some semantic models use different refresh controls depending on their storage mode and connection configuration.
Refreshing report visuals causes Power BI to retrieve query results according to the model's connection and caching behavior. It does not necessarily load new source rows into an Import-mode semantic model.
If imported data is outdated, the semantic model may need a data refresh before the report can display the latest source information.
Scheduled refresh allows Power BI Service to refresh supported semantic models automatically according to a defined schedule. It is especially useful for reports that must be updated at regular intervals.
Suppose a company receives updated sales transactions overnight and wants the report ready before the morning business meeting.
| Setting | Example |
|---|---|
| Semantic model | Sales Analytics Model |
| Refresh frequency | Daily |
| Preferred refresh time | 6:00 AM |
| Source | Sales SQL database |
| Connectivity | Configured cloud connection or gateway, depending on the source |
| Verification | Review refresh history and sales totals |
The example time is a planning choice, not a guarantee of the exact refresh completion time. Power BI may start or complete a scheduled refresh later than the requested time due to service resources or other constraints.
Scheduled refresh limits depend on the licensing and capacity configuration. Microsoft's documented standard limits include up to eight scheduled refreshes per day for Power BI Pro and up to 48 per day for eligible Premium per user and supported Premium or Fabric capacity scenarios. Other refresh operations and capacity-specific mechanisms may be governed by different rules.
Always confirm the current limits for the organization's license and model configuration before designing a refresh schedule.
Power BI can notify the semantic model owner about refresh failures and other relevant refresh events. Configure available notification settings and ensure that the owner monitors them.
Scheduled refresh may be paused after prolonged inactivity or disabled after repeated failures or certain unrecoverable errors. Resolve the underlying issue and re-enable the schedule when necessary.
A data gateway provides connectivity between Power BI Service and supported data sources that the cloud service cannot access directly. It is commonly used with on-premises databases and other protected network resources.
Power BI Desktop may be able to access a database through the local computer's network connection. After publishing, Power BI Service runs in the cloud and may not have direct access to that same database.
A configured gateway provides a supported route for the service to access the source for refresh or query operations.
Common gateway options include:
Power BI needs appropriate authentication to retrieve data from protected sources. A refresh may fail if credentials are missing, invalid, expired, or no longer authorized.
| Problem | Possible Cause | Solution |
|---|---|---|
| Authentication failure | Incorrect username, password, or authentication method | Verify credentials and select the supported authentication method. |
| Expired password | The source account password has changed or expired | Update the stored credentials according to organizational policy. |
| Access denied | The account lacks source-system permissions | Request the minimum required access from the source administrator. |
| OAuth sign-in failure | Expired authorization or changed organizational access | Reauthorize the supported connection and verify account access. |
| Gateway authentication error | Gateway data-source credentials are incorrect or unavailable | Ask the gateway administrator to verify the data-source definition and credentials. |
Incremental refresh is a feature for supported semantic models that refreshes a defined portion of a large dataset instead of reprocessing all historical data each time. It can improve refresh efficiency when a suitable date or other supported partitioning strategy is available.
Suppose a company stores five years of sales transactions containing millions of rows. The business usually needs to refresh recent transactions while retaining older historical data.
Instead of reloading every historical row on every refresh, a configured incremental refresh policy can maintain historical partitions and refresh a defined recent period, subject to the policy and supported source behavior.
Automatic page refresh updates visuals on an actively consumed report page at a configured interval in supported scenarios. It is primarily associated with DirectQuery and eligible capacity configurations.
Automatic page refresh is useful when users need to observe frequently changing information, such as operational metrics, live service activity, or near-real-time business indicators.
A customer support team may use a DirectQuery report to monitor the current number of open service tickets. Automatic page refresh can cause supported visuals to query for updated results at configured intervals while the report page is active.
Refresh history provides information about previous refresh attempts and helps identify whether a semantic model is updating successfully.
| Status | Meaning | Action |
|---|---|---|
| Completed | The refresh operation completed successfully. | Verify that the expected data is present. |
| Failed | The refresh operation did not complete successfully. | Review the error details and resolve the cause. |
| In progress | The refresh is still running. | Wait and monitor its status. |
| Cancelled | The refresh operation was cancelled. | Check whether cancellation was expected and retry if appropriate. |
| Warning or partial issue | The operation may have completed with a warning or an issue affecting part of the process. | Review the detailed status and confirm data completeness. |
Refresh errors can occur because of connectivity issues, source-system changes, invalid credentials, gateway failures, unsupported queries, or resource constraints.
| Error | Possible Cause | Solution |
|---|---|---|
| Credentials are invalid | Stored authentication details are incorrect or expired | Update the source credentials and test connectivity. |
| Gateway is offline | The gateway machine or service is unavailable | Restore gateway connectivity and verify its status. |
| Data source cannot be reached | Network outage, incorrect server name, or unavailable database | Check the connection details and confirm source availability. |
| Column not found | A source column was renamed or removed | Update Power Query transformations and model references, then republish as needed. |
| Data type conversion error | Source values do not match the expected data type | Correct the source values or adjust the transformation logic. |
| Query timeout | The source query takes too long | Optimize the query, reduce unnecessary transformations, or investigate source performance. |
| Memory or resource issue | The model or refresh exceeds available resources | Optimize the model and review capacity or refresh requirements. |
| Dynamic data source issue | The query uses a dynamic source configuration that is unsupported for the intended refresh scenario | Review the query design and use a supported data-source configuration. |
| Scheduled refresh is disabled | Repeated failures, inactivity, or an unrecoverable configuration error | Resolve the cause and re-enable the schedule if appropriate. |
| Data appears outdated | The refresh did not complete, the source has not updated, or visuals are showing cached results | Check source data, refresh history, storage mode, and visual refresh behavior. |
Refresh performance depends on the size and structure of the model, source-system performance, query design, network connectivity, gateway resources, and the capacity available to Power BI.
Models containing millions of rows may require significant processing time, especially when the full dataset is imported on every refresh.
Possible improvements include removing unnecessary columns, filtering irrelevant records, using incremental refresh where suitable, and optimizing the source queries.
Expensive joins, repeated transformations, and operations that prevent query folding can increase refresh duration.
Review the Applied Steps in Power Query Editor and simplify the query where possible.
A slow database or overloaded source system can delay refresh operations. Source indexing, query execution plans, server capacity, and concurrent workloads may all affect performance.
A gateway can become a bottleneck when it has insufficient resources, poor network connectivity, or too many demanding refresh operations.
Monitor gateway health and work with the gateway administrator to optimize or scale the deployment where needed.
Unnecessary columns, high-cardinality fields, complex calculated columns, and inefficient transformations can increase model size and processing requirements.
Data refresh must follow appropriate security and governance procedures. Power BI needs valid permissions to retrieve source information, and the resulting model must be protected against unauthorized access.
Refreshing a model does not automatically make its data secure. Workspace permissions, semantic model permissions, row-level security, object-level security, and source-system access must be configured according to the organization's requirements.
Refresh monitoring helps identify failures, unexpected data delays, and operational problems. Important business reports should have an assigned owner and a process for responding to refresh failures.
Consider a retail company that maintains sales information in a SQL database and uses Power BI to analyze revenue, profit, and regional performance.
| Metric | Before Refresh | After Successful Refresh |
|---|---|---|
| Total Sales | ₹5,00,000 | ₹5,50,000 |
| Total Orders | 500 | 550 |
| Reporting Period | Previous source update | Latest included source records |
| Refresh Status | Previous refresh completed | Completed successfully |
The figures above are illustrative. The actual results depend on the data available in the source, the query logic, and the refresh configuration.
The company can monitor sales performance using a report whose imported data is updated according to the configured refresh schedule. The owner can use refresh history to investigate failures and confirm whether the report reflects the expected data.
Exercise: Configure and test data refresh for a sample Power BI sales report.
After completing the exercise, learners should understand the difference between refreshing data in Desktop and Service, configuring scheduled refresh, checking refresh history, and diagnosing common refresh problems.
Answer: Data refresh updates data in a supported semantic model from its underlying sources so that reports can use the latest available information.
Answer: It helps keep imported data current, supports accurate reporting, and reduces the need for manual data updates.
Answer: Manual refresh is initiated on demand, while scheduled refresh runs automatically according to a configured schedule.
Answer: Import loads data into the semantic model, while DirectQuery generally sends queries to the underlying source when report visuals request results.
Answer: A data gateway provides a supported connection between Power BI Service and data sources that require access through an organization's network or gateway infrastructure.
Answer: Common reasons include invalid credentials, an offline gateway, source connectivity problems, schema changes, query errors, and resource constraints.
Answer: Incremental refresh updates a defined portion of a supported dataset instead of reprocessing all historical data during every routine refresh.
Answer: Refresh history records information about refresh attempts, including status, timing, duration, and available error details.
Answer: Not necessarily. Visual refresh updates query results, but an Import-mode semantic model generally needs a data refresh to load new source records.
Answer: Schema refresh synchronizes supported structural changes from the source, such as changes to columns or their definitions.
Answer: Automatic page refresh updates visuals on an active report page at a configured interval in supported scenarios, primarily with DirectQuery and eligible capacity settings.
Answer: Review refresh history, inspect error messages, verify credentials and gateway status, check source availability, and investigate query or schema changes.
Answer: The refresh may have failed, the source may not contain the expected update, the model may use a different storage mode, or the visuals may still be showing cached results.
Answer: Remove unnecessary columns, optimize transformations and source queries, preserve query folding where possible, use incremental refresh when suitable, and monitor resource usage.
Answer: Close & Apply applies pending Power Query changes and loads the resulting data into the model, while Refresh initiates a refresh operation on the existing model and its data.
Understanding Power BI Data Refresh is essential for maintaining reliable and accurate business intelligence reports. Refresh behavior depends on the storage mode, source system, semantic model, authentication, gateway connectivity, and service configuration.
Power BI Desktop is useful for refreshing and validating data during report development, while Power BI Service provides manual and scheduled refresh options for supported published semantic models. Additional features such as incremental refresh, automatic page refresh, and gateway connectivity help address different business and technical requirements.
By monitoring refresh history, managing credentials, optimizing data transformations, and configuring appropriate refresh schedules, Power BI developers can improve data freshness, reduce operational issues, and deliver more trustworthy insights.
