Extract nested column using another column value

Active3 hr before
Viewed126 times

2 Answers


APIs and document databases sometimes return nested JSON objects and you’re trying to promote some of those nested keys into column headers but loading the data into pandas gives you something like this:

Example_snippet/controller/utility/_using.js/ df = pd.DataFrame.from_records. . .
df = pd.DataFrame.from_records(results["issues"], columns = ["key", "fields"])

One possible way is to convert the struct type column to map type:, Questions ,Science Fiction & Fantasy, Podcast 377: You don’t need a math PhD to play Dwarf Fortress, just to code it

Example_snippet/controller/utility/_using.js/ import pyspark.sql.functions a. . .
import pyspark.sql.functions as F

df2 = df.withColumn(
F.from_json(F.to_json('data'), 'map<string,string>')[F.col('setting_name')]

Other "value-using" answers related to "Extract nested column using another column value"