In medium saucepan on medium-high heat, combine potatoes, salt and enough water to cover the potatoes. Cover and bring to a boil. Reduce heat to medium-low and simmer 10-15 minutes, or until desired tenderness.
While potatoes are cooking, prepare the dressing. In small bowl, whisk together olive oil, vinegar, parsley, garlic and anchovy paste. Season with salt and pepper, to taste. Set aside.
Drain the potatoes and allow to cool. In a medium, bowl, combine cooled potatoes, black olives, capers and dressing. Transfer to serving dish and serve at room temperature or chilled.