AssertionError: col should be Column

Question

Kenil Vasani

Asked: December 14, 20202020-12-14T21:03:59+00:00 2020-12-14T21:03:59+00:00In: Python

How to create a new column in PySpark and fill this column with the date of today?

This is what I tried:

import datetime
now = datetime.datetime.now()
df = df.withColumn("date", str(now)[:10])

I get this error:

AssertionError: col should be Column

You must login to add an answer.

Kenil Vasani · Accepted Answer · 2020-12-14T21:02:23+00:00

Best Answer

Kenil Vasani

How to create a new column in PySpark and fill this column with the date of today?

There is already function for that:

from pyspark.sql.functions import current_date

df.withColumn("date", current_date().cast("string"))

AssertionError: col should be Column

Use literal

from pyspark.sql.functions import lit

df.withColumn("date", lit(str(now)[:10]))