Source mapping for column unification and reordering

PatricioSanchez

Problem: 2 csv with same data but different order of the columns

We should be able to have 2 independent filters for different files in one connection and then merge them. Now I use 3 connections to solve it:
1) Filter to order the columns of 1st file

2) Filter to order the columns of 2nd file

3) Merge them

Karoline from Sheetgo

PatricioSanchez Hi Patricio,
How are you doing?

If I understood right:
For example, in this filter session, you want to have the option to select from which file are you choosing the columns.

PatricioSanchez

Karoline from Sheetgo thank you for your reply.

No, when I do a query to select which columns I want to transfer to the final destination. If I have 10 columns, I want 5 in a specific order.
The problem is when a 2nd file has the same name columns but in different order.

So, I have to do 2 queries in 2 different connections:

for the 1st file: “SELECT B, D, E, F, G”
for the 2nd file: “SELECT G, D, E, F, B”
and then create a 3rd connection to merge the result of these 2 previous results.

It would be great to filter each file independently in the same connection.

Karoline from Sheetgo

PatricioSanchez Hi Patricio!
I’ll be checking with my team and reach out to you soon! ;)

Eugenia

PatricioSanchez Unfortunately, we don’t currently have a way to create independent queries prior to merging.
The closest workaround would be to create individual connections that would do this mapping. Example:
Connection 1 -> File 1 to Google Sheets or CSV with Filter: Select B, D, E, F, G

Connection 2 -> File 2 to Google Sheets or CSV with Filter: SELECT G,D,E,F,B

Connection 3 -> Results from connections 1 and 2 Merged into a destination, no filter required.

@Willian from Sheetgo any other ideas?

Willian from Sheetgo

Eugenia I believe we can work on some functionality to solve PatricioSanchez use case.

If I got it correctly, in summary, he needs to reorder the columns for each of the source files, then, he would be able to do things like filtering, since the data would be unified within the same structure.

In this case, we need to architecture some sort of source mapping, this would be a type of data processor.

What do you think Eugenia?

Eugenia

Willian from Sheetgo
I believe that to be a good idea. We’ve had questions regarding “data mapping” a couple of times already.