Is there a way to compare two dataframes ?

[DestBro]

IDEA has this nice feature:

Sometimes I really want to compare two dataframes (for example excel/csv files; database exports, program output etc) and I want the same ease and usability like "Compare differences" feature from IDEA itself.

I need just to highlight the differences, something like:

Could be at plugin level or library level.

Also I noticed that the Notebook 'editor' itself does not have the "Compare with clipboard" feature in the context menu like other editors from the IDE:

It makes it even harder to compare anything inside the notebook environment.

[DestBro]

I would like ideally something like this:

or like this:

[koperagen]

Hi! I think there's no direct way to compare tables, however
With the help of Open in new tab button

you can split the editor (right click on a tab and "Split Right") and put two tables side by side:

[koperagen]

Is there lib directory under ide path? If not, it's not the IDE installation but something else. Check this instruction:
https://youtrack.jetbrains.com/articles/SUPPORT-A-528/How-to-find-where-IntellIJ-IDE-is-installed
Hope it helps!

[DestBro]

There is lib yes (also I tried all the IDEA paths I have on the computer)
It got it fixed by installing EAP version of IDEA

Now there are 2 other issues:

1. The stack-trace is related to the red block:
   

2. This API is not resolved by IDE, something is wrong with the dependency:
   

[koperagen]

Could it be that pathString is malformed? Judging by the log there are no path separators, which is odd. Maybe using / doesn't work for windows :(

[koperagen]

Try it.pathString.replace("\", "/") or it.pathString.replace("\", "\\")

[DestBro]

I tried to run it on Mac, it works as expected, many thanks! I hope it will get an official support in the notebook plugin one day, it's very handy.

[Jolanrensen]

We do have an open issue for a programmatical approach for comparing two dataframes #658. The idea is to have a function that performs something similar to a git diff, however this is quite an advanced topic we haven't come around to yet.

I do agree it would be nice to have the table comparing feature in notebooks. I'll ask the team about the possibilities :)

[DestBro]

Programmatically approach is also good, people can use it with Jupiter notebooks in browser etc, not just with Kotlin notebook plugin from IDEA as example, but it's also hard to achieve exactly the same thing as IDEA has with the "Compare feature": #1120 (reply in thread)

[DestBro]

Now after intellij-platform support was added you can write like this:

%use intellij-platform

import org.jetbrains.kotlinx.dataframe.*
import org.jetbrains.kotlinx.dataframe.api.*

val df1 = dataFrameOf("a")(1, 2, 3)
val df2 = dataFrameOf("a")(1, 2, 3)

import com.intellij.diff.DiffContentFactory
import com.intellij.diff.DiffDialogHints
import com.intellij.diff.DiffManager
import com.intellij.diff.contents.DiffContent
import com.intellij.diff.requests.SimpleDiffRequest
import com.intellij.openapi.application.runInEdt

runInEdt {
    val diffManager = DiffManager.getInstance()
    val contentFactory = DiffContentFactory.getInstance()

    val df1String = df1.toString()
    val df2String = df2.toString()

    // Create contents with dataframe string representations
    val leftContent: DiffContent = contentFactory.create(df1String)
    val rightContent: DiffContent = contentFactory.create(df2String)

    // Create diff request
    val diffRequest = SimpleDiffRequest(
        "DataFrame Comparison",
        leftContent,
        rightContent,
        "DataFrame 1",
        "DataFrame 2"
    )

    // Show diff window
    diffManager.showDiff(null, diffRequest, DiffDialogHints.FRAME)
}