pyspark.sql.functions.row_number

pyspark.sql.functions.row_number() → pyspark.sql.column.Column

Window function: returns a sequential number starting at 1 within a window partition.

New in version 1.6.0.

Changed in version 3.4.0: Supports Spark Connect.

Returns

Column

the column for calculating row numbers.

Examples

>>> from pyspark.sql import Window
>>> df = spark.range(3)
>>> w = Window.orderBy(df.id.desc())
>>> df.withColumn("desc_order", row_number().over(w)).show()
+---+----------+
| id|desc_order|
+---+----------+
|  2|         1|
|  1|         2|
|  0|         3|
+---+----------+

pyspark.sql.functions.rank pyspark.sql.functions.asc