Melt cols in Pandas by substrings of their names

The best way to convert the given dataframe into the desired format is to use the melt function. This function will allow the columns A_target_word_gd and A_target_word_fd to be combined into a single column called value_type, and the columns A_target_word_gd and B_target_word_gd to be combined into a single column called cond. The subject and subject_type columns will remain unchanged. The value column will contain the values from A_target_word_gd, A_target_word_fd, B_target_word_gd, and B_target_word_fd.

# melt dataframe
df_melt = df.melt(id_vars=['subject', 'subject_type'], var_name='value_type', value_name='value')

This code will produce the following dataframe:

     cond    subject    subject_type     value_type   value
      A         1        mild             gd           1           
      A         1        mild             fg           2           
      B         1        mild             gd           3            
      B         1        mild             fg           4  
      A         2        moderate         gd           11           
      A         2        moderate         fg           12           
      B         2        moderate         gd           13            
      B         2        moderate         fg           14          
...
# melt dataframe
df_melt = df.melt(id_vars=['subject', 'subject_type'], var_name='value_type', value_name='value')