Skip to main content

Advanced data transformation techniques using M language

Power BI's M language, also known as Power Query Formula Language, is a powerful tool for advanced data transformation. It allows you to manipulate and shape your data in ways that go beyond the capabilities of the standard Power BI user interface. Here are some advanced data transformation techniques using M language:


1. Custom Columns and Conditional Logic

You can create custom columns using complex logic. For example, you might want to create a new column based on multiple conditions.

m

Copy code

let

    Source = Excel.Workbook(File.Contents("C:\YourFile.xlsx"), null, true),

    Data = Source{[Name="Sheet1"]}[Data],

    AddCustom = Table.AddColumn(Data, "SalesCategory", each if [SalesAmount] > 500 then "High" else "Low")

in

    AddCustom

2. Pivot and Unpivot Data

M language allows for more complex pivoting and unpivoting of data.

Unpivot Example:

m

Copy code

let

    Source = Excel.Workbook(File.Contents("C:\YourFile.xlsx"), null, true),

    Data = Source{[Name="Sheet1"]}[Data],

    Unpivoted = Table.UnpivotOtherColumns(Data, {"ProductID", "ProductName"}, "Month", "SalesAmount")

in

    Unpivoted

Pivot Example:

m

Copy code

let

    Source = Excel.Workbook(File.Contents("C:\YourFile.xlsx"), null, true),

    Data = Source{[Name="Sheet1"]}[Data],

    Pivoted = Table.Pivot(Data, List.Distinct(Data[Month]), "Month", "SalesAmount", List.Sum)

in

    Pivoted

3. Merging Queries with Custom Join Logic

You can merge queries with custom join logic to combine tables based on more complex relationships.

m

Copy code

let

    Source1 = Excel.Workbook(File.Contents("C:\YourFile1.xlsx"), null, true),

    Table1 = Source1{[Name="Sheet1"]}[Data],

    Source2 = Excel.Workbook(File.Contents("C:\YourFile2.xlsx"), null, true),

    Table2 = Source2{[Name="Sheet2"]}[Data],

    MergedTables = Table.Join(Table1, {"KeyColumn1"}, Table2, {"KeyColumn2"}, JoinKind.FullOuter)

in

    MergedTables

4. Aggregations and Grouping

Perform aggregations and grouping using M language for more control over the operations.

m

Copy code

let

    Source = Excel.Workbook(File.Contents("C:\YourFile.xlsx"), null, true),

    Data = Source{[Name="Sheet1"]}[Data],

    Grouped = Table.Group(Data, {"Category"}, {{"TotalSales", each List.Sum([SalesAmount]), type number}})

in

    Grouped

5. Using Parameters and Custom Functions

Create parameters and custom functions for more dynamic data transformations.

Parameter Example:

m

Copy code

let

    paramYear = 2023,

    Source = Excel.Workbook(File.Contents("C:\YourFile.xlsx"), null, true),

    Data = Source{[Name="Sheet1"]}[Data],

    FilteredData = Table.SelectRows(Data, each [Year] = paramYear)

in

    FilteredData

Custom Function Example:

m

Copy code

let

    myFunction = (x as number) as number => x * x,

    Source = Excel.Workbook(File.Contents("C:\YourFile.xlsx"), null, true),

    Data = Source{[Name="Sheet1"]}[Data],

    AddCustom = Table.AddColumn(Data, "SquaredValue", each myFunction([Value]))

in

    AddCustom

6. Advanced Text Manipulation

M language allows for advanced text manipulations, such as splitting, replacing, and extracting parts of strings.

m

Copy code

let

    Source = Excel.Workbook(File.Contents("C:\YourFile.xlsx"), null, true),

    Data = Source{[Name="Sheet1"]}[Data],

    SplitColumn = Table.SplitColumn(Data, "FullName", Splitter.SplitTextByDelimiter(" "), {"FirstName", "LastName"}),

    ReplaceText = Table.ReplaceValue(SplitColumn, "OldValue", "NewValue", Replacer.ReplaceText, {"ColumnName"})

in

    ReplaceText

7. Error Handling

Handle errors in your data transformation process.

m

Copy code

let

    Source = Excel.Workbook(File.Contents("C:\YourFile.xlsx"), null, true),

    Data = Source{[Name="Sheet1"]}[Data],

    TryOperation = Table.AddColumn(Data, "SafeDivision", each try [Value1] / [Value2] otherwise null)

in

    TryOperation

8. Date and Time Transformations

Manipulate date and time data with M language.

m

Copy code

let

    Source = Excel.Workbook(File.Contents("C:\YourFile.xlsx"), null, true),

    Data = Source{[Name="Sheet1"]}[Data],

    AddDateParts = Table.AddColumn(Data, "Year", each Date.Year([Date])),

    AddMonth = Table.AddColumn(AddDateParts, "Month", each Date.Month([Date])),

    AddDay = Table.AddColumn(AddMonth, "Day", each Date.Day([Date]))

in

    AddDay


Conclusion

Using M language for data transformations in Power BI gives you extensive control over your data preparation processes. These advanced techniques enable you to create dynamic, flexible, and powerful data models that can handle complex scenarios. By mastering M language, you can significantly enhance your data transformation capabilities in Power BI

 


Comments

Post a Comment

Popular posts from this blog

Power BI tenant settings and admin portal

As of my last update, Power BI offers a dedicated admin portal for managing settings and configurations at the tenant level. Here's an overview of Power BI tenant settings and the admin portal: 1. Power BI Admin Portal: Access : The Power BI admin portal is accessible to users with admin privileges in the Power BI service. URL : You can access the admin portal at https://app.powerbi.com/admin-portal . 2. Tenant Settings: General Settings : Configure general settings such as tenant name, regional settings, and language settings. Tenant Administration : Manage user licenses, permissions, and access rights for Power BI within the organization. Usage Metrics : View usage metrics and reports to understand how Power BI is being used across the organization. Service Health : Monitor the health status of the Power BI service and receive notifications about service incidents and outages. Audit Logs : Access audit logs to track user activities, access requests, and administrative actions wit...
Post a Comment
Read more

Using bookmarks and buttons for navigation

Using bookmarks and buttons for navigation in Power BI allows you to create interactive experiences within your reports, guiding users through different views and sections. Let's walk through how to use bookmarks and buttons for navigation: Step 1: Create Bookmarks Navigate to the "View" tab : Open your report in Power BI Desktop and navigate to the "View" tab. Create Bookmarks : Select the elements (visuals, slicers, shapes, etc.) that you want to bookmark. Click on the "Bookmark" button in the "View" tab or right-click and select "Add bookmark". Name your bookmark and ensure the "Data" and "Display" options are selected if you want to capture filter states and visual display states. Repeat for Additional Views : Create bookmarks for each view or section of your report that you want to navigate to. Step 2: Create Buttons Insert Buttons : Go to the "Home" tab and click on the "Buttons" dropdow...
Post a Comment
Read more

Performance Optimization

Performance optimization in SQL is crucial for ensuring that your database queries run efficiently, especially as the size and complexity of your data grow. Here are several strategies and techniques to optimize SQL performance: Indexing Create Indexes : Primary Key and Unique Indexes : These are automatically indexed. Ensure that your tables have primary keys and unique constraints where applicable. Foreign Keys : Index foreign key columns to speed up join operations. Composite Indexes : Use these when queries filter on multiple columns. The order of columns in the index should match the order in the query conditions. Avoid Over-Indexing:  Too many indexes can slow down write operations (INSERT, UPDATE, DELETE). Only index columns that are frequently used in WHERE clauses, JOIN conditions, and as sorting keys. Query Optimization Use SELECT Statements Efficiently : SELECT Only Necessary Columns : Avoid using SELECT * ; specify only ...
Post a Comment
Read more